GCP Data Engineer Python Pyspark Developer (BigQuery, Cloud Storage, D

Preferred candidate profile

• GCP Services: BigQuery, Cloud Storage, Dataproc, Cloud Composer (managed Airflow) or self-managed Airflow.
• Airflow: Strong experience in DAG creation, operators/hooks, scheduling, backfilling, retry strategies, and CI/CD for DAG deployments.
• Programming: Proficiency in Python and Pyspark (PySpark, Airflow DAGs), SQL (advanced BigQuery SQL).
• Data Modeling: Dimensional modeling (Star/Snowflake), data vault basics, and schema design for analytics.
• Performance Tuning: BigQuery partitioning/clustering, predicate pushdown, job stats review, Dataproc executor tuning.
• Version Control & CI/CD: Git, branching strategies, pipelines for deploying Airflow DAGs and config.
• Operational Excellence: Monitoring with Stackdriver/Cloud Logging, debugging pipeline failures, and root-cause analysis.
• involves end-to-end ownership of data ingestion, transformation, orchestration, and performance tuning for batch and near real-time workflows.