Data Engineer

Data Engineer

Experience: 3+ Years

Location: Bengaluru(Bengaluru based candidates preferred for a F2F Interview)

Immediate Joiners

Candidates should be available for the F2F Interview.

About the role:

We are looking for a skilled Data Engineer with 3+ years of experience in building and maintaining scalable data pipelines and data platforms. The ideal candidate should have hands-on expertise in Apache Spark, SQL, Python, and Microsoft Azure services, with a strong understanding of data engineering best practices and cloud-based data solutions.

Key Responsibilities

• Design, develop, and optimize scalable ETL/ELT data pipelines using Spark and Python.
• Build, maintain, and enhance data ingestion frameworks for structured and unstructured data.
• Write efficient and optimized SQL queries for data extraction, transformation, and reporting.
• Develop and manage data workflows and pipelines on Microsoft Azure.
• Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements.
• Monitor data pipelines, troubleshoot production issues, and ensure data quality, reliability, and availability.
• Implement data validation, logging, monitoring, and alerting mechanisms.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Work with large-scale datasets in distributed computing environments.
• Follow data governance, security, privacy, and compliance standards.
• Participate in code reviews and contribute to continuous improvement of engineering practices.

Required Skills

• 3 + years of experience in Data Engineering.
• Strong hands-on experience with Apache Spark (PySpark or Spark SQL).
• Proficiency in Python for data processing, automation, and application development.
• Advanced SQL skills with experience in query optimization and performance tuning.
• Hands-on experience with Microsoft Azure data serviceS
• Experience building batch and/or streaming data pipelines.
• Knowledge of data modeling and data warehousing concepts.
• Experience with Git and CI/CD practices.
• Strong analytical, problem-solving, and debugging skills.