Data Engineer

This role is primarily focused on SQL and Spark. Please apply only if you have strong hands-on experience in both SQL and Spark.

Key Responsibilities

Pipeline Development: Design, build, and optimize robust ETL/ELT data pipelines using Python, SQL, and Apache Spark to process large volumes of structured and unstructured data.

Cloud Infrastructure: Implement and manage data solutions on at least one major cloud platform (AWS, GCP, or Azure).

Data Modeling: Architect efficient data models (dimensional modeling, star/snowflake schemas) for data lakes, data warehouses, and reporting layers.

Performance Tuning: Monitor, troubleshoot, and optimize data pipelines and queries for maximum performance and cost-efficiency.

Collaboration & Delivery: Work directly with cross-functional and client-facing teams to gather requirements, deliver robust data solutions, and follow best engineering practices (CI/CD, version control, testing).

Required Skills & Qualifications

Experience: 2 to 4 years of hands-on software/data engineering experience, preferably within a client-facing or IT services environment.

Core Languages: Advanced proficiency in SQL (complex queries, performance tuning) and Python (scripting, data manipulation libraries).

Data Frameworks: Strong practical experience with Apache Spark (PySpark) for distributed data processing.

Cloud Platforms: Hands-on implementation experience with any one cloud ecosystem:

AWS (S3, Glue, EMR, Redshift, Athena)

GCP (BigQuery, Dataflow, Dataproc, GCS)

Azure (Azure Data Factory, Databricks, Synapse Analytics)

Data Concepts: Solid understanding of data warehousing concepts, ETL/ELT architecture, workflow orchestration tools (e.g., Apache Airflow), and data governance.

Good-to-Have Skills

Experience with modern data stack tools (dbt, Snowflake).

Containerization and orchestration tools like Docker and Kubernetes.

Exposure to streaming data technologies (Kafka, Spark Streaming).