Gcp Data Engineer

Role & responsibilities

• Design, develop, and optimize scalable data pipelines using GCP services.
• Build and maintain batch and real-time data ingestion and processing frameworks.
• Develop enterprise-grade data warehousing solutions using BigQuery.
• Implement ETL/ELT processes for structured and unstructured data.
• Integrate data from multiple sources including databases, APIs, files, and streaming platforms.
• Ensure data quality, governance, security, and compliance across data platforms.
• Optimize query performance, storage, and processing costs within GCP.
• Automate deployment and operational processes using CI/CD and Infrastructure as Code (IaC).
• Monitor data pipelines and troubleshoot production issues.
• Collaborate with Data Architects, Business Analysts, and Data Scientists to translate business requirements into technical solutions.
• Implement data models, metadata management, and data lineage best practices.
• Support migration of on-premise or multi-cloud data platforms to GCP.
• Mentor junior engineers and participate in technical design reviews.

• Google Cloud Storage (GCS)
• BigQuery
• Cloud Composer (Apache Airflow)
• Dataflow
• Dataproc
• Pub/Sub
• Cloud Functions
• Cloud Run
• Cloud SQL
• BigLake (preferred)
• Vertex AI integration (good to have)
• Python
• SQL (Advanced)
• PySpark