Data Engineer II

We are looking for a hands-on Data Engineer II with strong Google Cloud Platform experience to build scalable data pipelines, transformations, and reusable data-platform capabilities. You will work across data engineering and data platform engineering, independently delivering production-grade data products while contributing reusable components and engineering patterns that can be adopted across multiple teams and use cases. The role requires strong hands-on development skills and the ability to take a solution from design through implementation, testing, deployment, and production support.

Responsibilities:

• Design, develop, and maintain production-grade data pipelines on Google Cloud Platform.
• Build scalable batch and incremental data-processing solutions.
• Develop complex transformations using BigQuery, SQL, and Dataform/dbt.
• Build and maintain data-processing workflows using Cloud Composer / Apache Airflow.
• Develop reusable transformation components, orchestration patterns, libraries, and templates.
• Implement metadata-driven and configuration-driven processing where appropriate.
• Build distributed data-processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark.
• Design data models for analytical and downstream data-product requirements.
• Implement incremental and idempotent processing patterns.
• Define and maintain schemas and data contracts.
• Implement automated data-quality checks, validation, and reconciliation.
• Capture and integrate metadata and lineage into data-processing workflows.
• Optimize BigQuery queries and pipelines for performance and cloud cost.
• Implement automated testing and integrate data workloads with CI/CD pipelines.
• Build monitoring and operational controls for production pipelines.
• Troubleshoot production issues and perform root-cause analysis.
• Contribute to reusable data-platform capabilities and engineering standards.
• Collaborate with data engineers, platform engineers, DevOps, architects, and business stakeholders.

Requirements:

• 4+ years of hands-on data engineering experience building production data solutions.
• Strong practical experience with Google Cloud Platform (GCP).
• Strong hands-on experience with BigQuery.
• Advanced SQL skills, including complex joins, CTEs, window functions, and query tuning.
• Strong Python development skills for data processing, automation, and testing.
• Experience developing production ETL / ELT pipelines.
• Hands-on experience with Cloud Composer / Apache Airflow.
• Experience with Dataform and/or dbt.
• Experience designing batch and incremental processing pipelines.
• Experience with Dataflow / Apache Beam or Dataproc / Spark / PySpark.
• Strong understanding of data modelling, including normalization, denormalization, and dimensional modelling.
• Experience working with large-scale datasets.
• Experience with incremental and idempotent pipeline patterns.
• Understanding of data contracts and schema evolution.
• Experience implementing data-quality validation and reconciliation.
• Understanding of metadata and data lineage.
• Experience with Git, automated testing, and CI/CD.
• Experience troubleshooting and supporting production data pipelines.
• Ability to independently design solutions rather than only implement predefined specifications.

Technical Skills:

• Cloud and Storage: Google Cloud Platform, BigQuery, Google Cloud StorageAirflow.
• Transformation and Orchestration: Advanced SQL, Dataform/dbt, Cloud Composer/Apache Airflow.
• Data Processing: Dataflow / Apache Beam, Dataproc / Spark / PySpark.
• Programming and Engineering: Python, Git, automated testing, CI/CD, monitoring, and troubleshooting.
• Data Engineering Capabilities: ETL/ELT, batch processing, incremental and idempotent pipelines, data modelling, data contracts, data quality and reconciliation, metadata and lineage, query performance optimization, cloud-cost optimization, and reusable data-platform components.

Good to Have:

• Experience with Apache Iceberg or modern lakehouse table formats.
• Experience with streaming or event-driven processing.
• Practical understanding of Data Product / Data Mesh principles.
• Experience with metadata-driven or configuration-driven processing.
• Experience with Change Data Capture.
• Familiarity with Terraform / Infrastructure as Code.
• Experience building reusable components used by multiple engineering teams.
• Strong Interpersonal Skills
• Strong ownership mindset and ability to take engineering work through production.
• Strong analytical, debugging, and problem-solving skills.
• Ability to communicate technical decisions clearly.
• Comfortable participating in design and code reviews.
• Ability to collaborate with engineering and business stakeholders.
• Ability to work independently while seeking guidance for complex architectural decisions.
• Comfortable working within distributed and multicultural teams.