Data Engineer

Data Engineer

4 - 8 Years of Experience

Location: Bengaluru (Preferred Bengaluru based candidates to apply)

Should be available to attend the Face to Face Interview

Immediate Joiners Preferred

About the Role

We are looking for a skilled Data Engineer with 4 - 8 years of experience in building and maintaining scalable data pipelines and data platforms. The ideal candidate should have hands-on expertise in Spark, SQL, Python with a strong understanding of data engineering best practices and cloud-based data solutions.

Key Responsibilities

• Design, develop, and optimize scalable ETL/ELT data pipelines using Spark and Python.
• Build, maintain, and enhance data ingestion frameworks for structured and unstructured data.
• Write efficient and optimized SQL queries for data extraction, transformation, and reporting.
• Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements.
• Monitor data pipelines, troubleshoot production issues, and ensure data quality and reliability.
• Implement data validation, logging, and monitoring mechanisms.
• Optimize Spark jobs for performance and cost efficiency.
• Work with large-scale datasets in distributed computing environments.
• Follow data governance, security, and compliance standards.
• Participate in code reviews and contribute to continuous improvement of engineering practices.

Required Skills

• 4 - 8 years of experience in Data Engineering.
• Strong hands-on experience with Apache Spark (PySpark or Spark SQL).
• Proficiency in Python for data processing and automation.
• Advanced SQL skills with experience in query optimization.
• Experience in building batch and/or streaming data pipelines.
• Knowledge of data modeling and data warehousing concepts.
• Experience with Git and CI/CD practices.
• Strong analytical, problem-solving, and debugging skills.

Preferred Qualifications

• Experience with Apache Airflow or similar workflow orchestration tools.
• Exposure to Kafka or other streaming technologies.
• Familiarity with Delta Lake, Apache Iceberg, or Apache Hudi.
• Experience with Infrastructure as Code (Terraform or CloudFormation) is a plus.
• AWS Certification is an added advantage.