Senior Data Engineer II

We are looking for a Data Engineer II with strong hands-on experience in GCP, BigQuery, Python, SQL, and modern data engineering technologies to build and maintain scalable, reliable, and production-grade data platforms. The ideal candidate should have strong expertise in data pipeline development, ETL/ELT, data modelling, pipeline optimization, and production support, with the ability to work effectively with large-scale datasets on GCP.

Responsibilities:

• Design, develop, and maintain scalable data pipelines on GCP.
• Work with large-scale datasets using BigQuery and other GCP data services.
• Develop robust ETL/ELT pipelines for batch and incremental data processing.
• Implement idempotent and reliable data processing workflows.
• Build and maintain workflows using Airflow / Cloud Composer.
• Develop data transformation pipelines using Dataform/dbt.
• Work with Dataflow / Apache Beam for distributed data processing.
• Work with Dataproc, Spark, and PySpark for large-scale data processing.
• Optimize data pipelines for performance, scalability, and cost efficiency.
• Implement data quality checks, testing, and monitoring to ensure reliable data products.
• Troubleshoot production issues and provide effective production support.
• Collaborate with engineering and business teams to deliver high-quality data solutions.

Requirements:

• Strong hands-on experience with GCP and BigQuery.
• Advanced SQL and Python, Airflow / Cloud Composer, Dataform / dbt, Dataflow / Apache Beam, Dataproc / Spark / PySpark.
• Strong understanding of ETL/ELT and data pipeline development.
• Experience with batch, incremental, and idempotent processing.
• Good understanding of data modelling and data quality.
• Experience in pipeline optimization and troubleshooting.
• Hands-on experience with Git, CI/CD, and testing.
• Strong production support and problem-solving skills.
• 4-7.5 years of relevant data engineering experience.
• Strong experience building production-grade data pipelines.
• Excellent SQL and Python skills.
• Strong hands-on experience with the GCP data ecosystem, particularly BigQuery.
• Ability to work with large datasets and optimize data processing workflows.
• Strong analytical and troubleshooting skills.
• Good understanding of software engineering best practices, including version control, CI/CD, and testing.
• Strong communication and collaboration skills.