Data Engineer

Role Overview -

We are looking for a GCP Data Engineer with 2-3 years of experience in data engineering and cloud-based development. The ideal candidate should have hands-on experience with Python, SQL/PLSQL, Google Cloud Platform (GCP), BigQuery, Data Fusion, and Cloud Storage.

The role involves developing and maintaining scalable data pipelines and cloud-based data solutions while working with large datasets and modern GCP services.

Key Responsibilities -

• Develop and maintain data engineering solutions on GCP.
• Write, optimize, and debug SQL/PLSQL queries for data processing and analytics.
• Develop data pipelines using BigQuery, Data Fusion, and Cloud Storage.
• Use Python for data processing, automation, and integration.
• Work with Cloud Functions, Cloud Run, and/or App Engine.
• Develop data ingestion, transformation, and integration workflows.
• Process and transform structured and semi-structured data.
• Troubleshoot data pipelines and cloud-based applications.
• Monitor pipeline performance and assist with performance optimization.
• Follow best practices around scalability, security, reliability, and maintainability.
• Write unit tests and technical documentation.
• Participate in code reviews and technical discussions.
• Work with SQL and NoSQL databases as required.
• Collaborate with data engineers and cross-functional teams.

Required Skills -

• 2-3 years of experience in Data Engineering / Cloud Data Engineering.
• Hands-on experience with Google Cloud Platform (GCP).
• Strong programming experience in Python.
• Hands-on experience with BigQuery.
• Experience with GCP Data Fusion and data pipeline development.
• Strong knowledge of SQL and PL/SQL.
• Experience with Google Cloud Storage (GCS).
• Experience with one or more of:
• Google Cloud Functions
• Cloud Run
• App Engine
• Understanding of ETL/ELT and data integration concepts.
• Good understanding of relational and NoSQL databases.
• Experience working with large datasets.
• Strong troubleshooting and problem-solving skills.

Good to Have -

• Knowledge of GCP IAM, security, monitoring, and logging.
• Exposure to CI/CD and DevOps practices.
• Understanding of BigQuery performance tuning and cost optimization.
• Basic knowledge of data warehouse architecture and dimensional modelling.
• Knowledge of Agile/Scrum methodologies.