Apache Spark E2

Job Title

Apache Spark Engineer (E2)

Experience

4 to 8 Years

Location

Hyderabad / Bangalore / Pune / Chennai / Gurgaon / Kolkata

Job Summary

We are looking for a skilled Apache Spark Engineer with strong experience in distributed data processing, big data technologies, and cloud platforms. The candidate will be responsible for designing, developing, and optimizing large-scale data pipelines and processing frameworks using Apache Spark. Relevant Spark and Data Engineering requirements are reflected in internal Data Engineer references. [tcscomprod...epoint.com], [tcscomprod...epoint.com]

Required Skills

• Apache Spark (Core, SQL, Streaming)
• PySpark / Scala Spark
• Python Programming
• Hadoop Ecosystem
• SQL and Data Modeling
• ETL Development
• Data Warehousing Concepts
• Kafka (Preferred)
• Airflow / Scheduler Tools
• Cloud Platforms (GCP/AWS/Azure)
• Git and CI/CD
• Linux/Unix

Key Responsibilities

• Develop and optimize Spark-based batch and streaming applications.
• Design scalable ETL data pipelines.
• Process large structured and unstructured datasets.
• Work with business and data teams to gather requirements.
• Implement data quality and monitoring frameworks.
• Optimize Spark jobs for performance and cost.
• Support cloud-based data platform modernization.
• Maintain documentation and operational procedures.

Good to Have

• Databricks
• Delta Lake
• Kafka Streaming
• Docker & Kubernetes
• Data Engineering Certifications

2. Digital : Google Data Engineering E2

Job Title

Google Data Engineer (GCP Data Engineering) E2

Experience

4 to 8 Years

Location

Hyderabad / Bangalore / Pune / Chennai / Gurgaon / Kolkata

Job Summary

We are seeking a Google Cloud Data Engineer to build and maintain scalable data platforms on GCP. The role involves data ingestion, transformation, orchestration, and analytics using native GCP services. Enterprise data engineering references include experience with orchestration frameworks, cloud workflows, Python, and large-scale data processing. turn2search25

Required Skills

• Google Cloud Platform (GCP)
• BigQuery
• Dataflow
• Cloud Compose (Airflow)
• Cloud Storage
• Pub/Sub
• Python
• SQL
• ETL/ELT Development
• Data Modeling
• CI/CD Concepts
• Git

Preferred Skills

• Apache Spark / PySpark
• Dataproc
• Cloud Functions
• Terraform
• Kafka
• dbt
• Vertex AI Exposure

Key Responsibilities

• Design and implement scalable data pipelines on GCP.
• Develop ingestion and transformation workflows.
• Build and optimize BigQuery data models.
• Implement orchestration frameworks using Composer/Airflow.
• Work with structured, semi-structured, and streaming data.
• Monitor performance, reliability, and data quality.
• Support analytics and reporting requirements.
• Collaborate with architects, analysts, and business stakeholders.

Qualifications

• Bachelor's degree in Computer Science, IT, or equivalent.
• Experience with enterprise-scale cloud data platforms.
• Strong problem-solving and analytical skills.
• Excellent communication and stakeholder management skills.

Naukri Search Keywords

Apache Spark E2

Apache Spark, PySpark, Spark SQL, Spark Streaming, Scala, Hadoop, Kafka, Airflow, ETL Developer, Data Engineer, Databricks, Delta Lake, Python, Big Data Engineer

Google Data Engineering E2

GCP Data Engineer, Google Cloud Platform, BigQuery, Dataflow, Dataproc, Cloud Composer, Airflow, PubSub, Python, SQL, ETL, Data Modeling, Cloud Functions, Terraform, dbt, Data Engineer,