Data Engineer

Position :- Big Data Engineer

Location :- Bangalore / Chennai

Experience 4 +Years

Job Summary:
We are looking for a skilled Big Data Engineer to design, develop, and maintain scalable data processing solutions. The ideal candidate should have strong experience with Python/Java/Scala, Spark, Hadoop, SQL, ETL pipelines, and cloud platforms.

Key Responsibilities

• Design and develop scalable Big Data processing and ETL/ELT pipelines.
• Develop data processing solutions using Apache Spark / PySpark.
• Work with large and complex datasets to perform data transformation and analysis.
• Build and optimize data pipelines using Python, Java, or Scala.
• Develop complex SQL queries and optimize database performance.
• Work with Hadoop ecosystem components such as HDFS, Hive, Kafka, and YARN.
• Implement real-time and batch data processing solutions.
• Integrate data from multiple sources into data lakes/data warehouses.
• Monitor, troubleshoot, and optimize data pipelines and jobs.
• Ensure data quality, reliability, security, and performance.
• Collaborate with Data Scientists, Data Analysts, Architects, and business teams.
• Participate in code reviews, testing, deployment, and production support.
• Apache Iceberg is mandatory skill.

Required Skills

• Strong programming experience in Python / Java / Scala.
• Hands-on experience with Apache Spark / PySpark.
• Strong SQL skills.
• Experience with ETL/ELT and data pipeline development.
• Good understanding of Hadoop ecosystem.
• Experience with Hive, Kafka, HDFS, YARN.
• Experience with Data Lake / Data Warehouse concepts.
• Knowledge of Git, CI/CD, and DevOps practices.
• Good understanding of distributed computing and data processing.