GCP Data Engineer - Vertex AI

Business Title :

GCP Data Engineer Vertex AI

Years of Experience: 4 to 7 years

Shift timing

12 PM - 10 PM

Location : Mumbai, Pune, Bangalore

Immediate to 20 Days Joiners only.

Key responsibilities:

Creates and maintains optimal data pipeline architecture

Assembles large, complex data sets that meet functional / non-functional business requirements

Identifies, designs and implements internal process improvements: automating manual processes, optimising

Title: GCP Data engineer

Location - Pune, Bengaluru, Mumbai

About the Role

We are seeking a highly motivated and hands-on GCP Data Engineer to support the development and delivery of modern enterprise data and AI platforms on Google Cloud Platform (GCP).

This role will work closely with Lead Data Engineers, Enterprise Architects, analytics teams, and AI/ML teams to build scalable, reliable, and AI-ready data pipelines and cloud-native data solutions that support enterprise analytics, intelligent automation, and digital transformation initiatives.

The ideal candidate should possess strong foundations in cloud data engineering, distributed data processing, and modern data platform development, along with a passion for building high-quality, scalable, and reusable data solutions in a fast-paced enterprise environment.

This role offers an excellent opportunity to work on large-scale data modernization, semantic data enablement, and next-generation AI data ecosystems.

Key Responsibilities

1. Cloud Data Engineering & Pipeline Development

• Develop and maintain scalable batch and real-time data pipelines on GCP.
• Build ingestion, transformation, and serving pipelines supporting enterprise analytics and AI use cases.
• Assist in modernization of legacy data workflows into cloud-native architectures.
• Develop reusable and maintainable data engineering components following established architectural standards.
• Support implementation of event-driven and streaming-based data processing solutions.

2. Data Product Development

• Contribute to development of reusable and domain-oriented data products.
• Implement data transformation logic and standardized data models supporting downstream analytics and AI consumption.
• Support implementation of:
• Data quality validations
• Schema management
• Metadata enrichment
• Data contracts
• Reusable transformation frameworks
• Ensure data pipelines are reliable, scalable, and production-ready.

3. GCP Platform Development

• Work with GCP-native services including:
• BigQuery
• Dataflow
• Dataproc
• DBT
• Pub/Sub
• Cloud Storage
• Cloud Composer (Airflow)
• Cloud SQL
• Develop ETL/ELT pipelines and optimize data processing workloads.
• Support orchestration and scheduling of enterprise data workflows.
• Monitor and troubleshoot pipeline performance, failures, and operational issues.

4. Semantic & Analytics Enablement

• Support implementation of semantic models and business-friendly data structures for analytics and reporting.
• Collaborate with analytics and BI teams to improve consistency and usability of enterprise data assets.
• Assist in development of standardized metrics, dimensions, and reusable reporting datasets.
• Contribute to metadata and data catalog integration initiatives.

5. AI/ML Data Enablement

• Build and optimize AI-ready data pipelines supporting ML and GenAI initiatives.
• Support feature engineering and data preparation workflows for AI/ML use cases.
• Assist in integration with:
• Vertex AI
• BigQuery ML
• Vector databases
• GenAI frameworks
• Contribute to implementation of semantic search and AI-assisted data interaction patterns.

6. Engineering Best Practices & Collaboration

• Follow established coding standards, architecture guidelines, and DevOps practices.
• Participate in code reviews, testing, debugging, and performance optimization activities.
• Collaborate effectively with architects, lead engineers, analysts, and client stakeholders.
• Contribute to engineering documentation, operational runbooks, and technical knowledge sharing.
• Continuously learn and adopt modern cloud, data engineering, and AI platform technologies.

7. Governance, Monitoring & Operational Support

• Support implementation of monitoring, logging, lineage, and observability frameworks.
• Ensure adherence to enterprise security, governance, and compliance standards.
• Assist in incident resolution, root cause analysis, and platform stability improvements.
• Contribute to continuous improvement initiatives for operational excellence and delivery quality.

Technical Expertise Required

Area

Skills / Technologies

Cloud Data Engineering

GCP, BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage

Data Processing

SQL, Python, PySpark, DBT

Streaming & Pipelines

Apache Beam, batch & real-time processing

Workflow Orchestration

Cloud Composer (Airflow), Workflows

Semantic & Analytics

Basic semantic modeling concepts, reporting datasets, Looker exposure preferred

AI/ML Enablement

Vertex AI exposure, BigQuery ML, GenAI ecosystem awareness

Metadata & Governance

Data Catalog, metadata management, lineage concepts

DevOps & Automation

CI/CD concepts, Git, automation frameworks

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.
• 3–6 years of experience in data engineering and cloud-based data platform development.
• Hands-on experience working with Google Cloud Platform (GCP) data services.
• Strong SQL and Python programming skills.
• Experience developing scalable ETL/ELT pipelines and distributed data processing workflows.
• Understanding of modern data architecture concepts including data lakes, data warehouses, and streaming pipelines.
• Exposure to analytics, AI/ML, or GenAI-enabled data ecosystems preferred.
• Strong analytical, troubleshooting, and problem-solving skills.
• Ability to work collaboratively in Agile and cross-functional delivery teams.
• GCP certifications such as Associate Cloud Engineer or Professional Data Engineer are a plus.