Gcp Data Engineer

Job Title: Data Engineer
Location: Hyderabad
Experience Level: 3-8 Years

About the Role:

We are seeking a highly skilled and experienced Senior Data Engineer to join our growing team in Pune. The ideal candidate will have a strong background in data engineering, with a particular focus on Google Cloud Platform (GCP), Apache Airflow, and end-to-end ETL pipeline development. You will be responsible for designing, developing, and maintaining robust and scalable data pipelines, ensuring data quality, and optimizing data solutions for performance and cost. This role requires a hands-on approach and the ability to work both independently and collaboratively in an agile environment.

Responsibilities:

• Design, develop, and deploy scalable ETL pipelines using GCP data services, including BigQuery, Cloud Composer (Apache Airflow), and Cloud Storage.
• Develop, deploy, and manage complex DAGs in Apache Airflow for orchestrating data workflows.
• Write and optimize complex SQL and PL/SQL queries, stored procedures, and functions for data manipulation, transformation, and analysis.
• Optimize BigQuery workloads for performance, cost efficiency, and scalability.
• Develop scripts using Python and Shell scripting to support automation, data movement, and transformations.
• Ensure data quality, integrity, and reliability across all data solutions.
• Collaborate with cross-functional teams including data scientists, analysts, and engineers to understand data requirements and deliver effective solutions.
• Participate in code reviews and contribute to establishing and maintaining data engineering best practices.
• Troubleshoot and resolve data pipeline issues in a timely manner.
• Use version control systems (e.g., Git) for managing code and collaborating on engineering work.
• Stay updated on the latest trends and technologies in data engineering, cloud computing, and ETL processes.

Required Skills and Qualifications:

• 38 years of hands-on experience in Data Engineering roles.
• Mandatory Skills:
• ETL pipeline design and implementation
• SQL and PL/SQL (complex queries, procedures, and transformations)
• Google Cloud Platform (GCP) — BigQuery, Cloud Composer, Cloud Storage
• Apache Airflow (designing and deploying complex DAGs)
• Python for scripting and data processing
• Shell Scripting for automation and orchestration tasks
• Experience with Informatica is a strong plus.
• Proven ability to optimize BigQuery for performance and cost.
• Familiarity with Git and version control best practices.
• Excellent problem-solving, analytical, and communication skills.
• Ability to thrive both independently and within a collaborative agile team.
• Bachelor’s degree in Computer Science, Engineering, or a related field.

What We Offer:

• A challenging and rewarding role in a dynamic and fast-paced environment.
• Opportunity to work with cutting-edge technologies on the Google Cloud Platform.
• A collaborative and supportive team culture.
• Continuous learning and professional growth opportunities.