GCP Data Engineer

GCP Data Engineer | 6-8 Years of experience | Chennai

Required Technical Skills

• Python
• SQL
• PySpark
• GCP (Preferred) or Azure
• Snowflake
• Airflow
• Jenkins / Jira / Git
• CI/CD
• Kubernetes / Docker

Must-Have

• Strong development experience in Python, PySpark, and SQL.
• Experience with cloud platforms (GCP preferred / Azure).
• Experience working in an Agile delivery environment.
• Experience with orchestration tools such as Airflow and ADF.
• Experience with GCP services:
• GCS Bucket
• Cloud Functions
• Dataproc
• Dataflow
• Pub/Sub
• BigQuery
• Hands-on experience in Snowflake.
• Experience building automated data pipelines.
• Experience in data analysis and data exploration.
• Experience working in a multi-developer environment using version control tools such as Git.
• Strong critical thinking, communication, and problem-solving skills.

Good-to-Have

• Understanding of DevOps best practices and CI/CD.
• Understanding of containerization technologies such as Kubernetes and Docker.
• Healthcare domain knowledge.

Key Responsibilities

• Perform data analytics on large-scale datasets to solve complex business problems and recommend solutions.
• Develop and enhance end-to-end data pipelines using big data technologies such as Hadoop, Cloud platforms, and the Spark ecosystem.
• Troubleshoot complex data structures and perform root cause analysis to proactively resolve operational and product issues.
• Develop and document test cases, execute testing, and share UAT results as part of Agile sprints.
• Perform end-to-end data pipeline integration testing in compliance with change management processes and standards.
• Adhere to coding, design, and application standards, including Python and SQL best practices.
• Leverage DevOps practices for production deployments, including CI/CD, test-driven development, and version control using Jenkins, Jira, and Git.
• Collaborate with business stakeholders to define product objectives, business requirements, system designs, and key performance indicators (KPIs).