Data Engineer

Role & responsibilities

Assess the candidate for a senior Data Engineering role with strong emphasis on SQL, PySpark, and Python. Evaluate their ability to design and implement scalable batch and distributed data pipelines, optimise complex SQL queries on distributed engines, and build high-performance Spark-based transformations using DataFrames and Spark SQL. Validate hands-on expertise in Python for developing reusable data frameworks, automation, and production-grade components, along with experience in lakehouse architectures (Delta/Iceberg) and cloud storage (S3/ADLS). Probe their understanding of ETL/ELT design patterns, data modelling (dimensional/star schema), data quality frameworks, and orchestration tools like Airflow. Go beyond technical depth to assess problem-solving ability, debugging approach, and performance optimisation mindset through real scenarios. Evaluate ownership, and stakeholder handling through situational questions. Look for a Forward Deployed Engineer mindsetability to handle ambiguity, iterate quickly, and deliver pragmatic solutions. The candidate should demonstrate strong coding fluency, structured thinking, and a balance of engineering rigour with street-smart execution. Ensure there is a medium-complex hands-on coding exercise questions on real-world problems and assess the coding for logical approach, coding standards, maintainability and performance optimizations used in the answers