Data Engineer

Job Title:

Data Engineer

Experience Range:

6-9 years

Location : Bengaluru

Notice period : immediate - 15 days

Job Summary:

We are looking for a skilled Data Engineer to design, build, and maintain robust, scalable data pipelines and infrastructure. This role is essential in enabling data accessibility, quality, and insights across the organization. You will work with modern cloud and big data technologies such as Azure Databricks, Snowflake, and DBT, collaborating with cross-functional teams to power data-driven decision-making.

Key Responsibilities:

For External Candidates:

• Data Pipeline Development:
  Build and optimize data pipelines to ingest, transform, and load data from multiple sources using Azure Databricks, Snowflake, and DBT.
• Data Modeling & Architecture:
  Design efficient data models and structures within Snowflake ensuring optimal performance and accessibility.
• Data Transformation:
  Implement standardized and reusable data transformations in DBT for reliable analytics and reporting.
• Performance Optimization:
  Monitor and tune data workflows for performance, scalability, and fault-tolerance.
• Cross-Team Collaboration:
  Partner with data scientists, analysts, and business users to support analytics and machine learning projects with reliable, well-structured datasets.

Additional Responsibilities (Internal Candidates):

• Implement and manage CI/CD pipelines using tools such as Jenkins, Azure DevOps, or GitHub.
• Develop data lake solutions using Scala and Python in a Hadoop/Spark ecosystem.
• Work with Azure Data Factory and orchestration tools to schedule, monitor, and maintain workflows.
• Apply deep understanding of Hadoop architecture, Spark, Hive, and storage optimization.

Mandatory Skills:

• Hands-on experience with:
  
• Azure Databricks (data processing and orchestration)
• Snowflake (data warehousing)
• DBT (data transformation)
• Azure Data Factory (pipeline orchestration)
• Strong SQL and data modeling capabilities
• Proficiency in Scala and Python for data engineering use cases
• Experience with big data ecosystems: Hadoop, Spark, Hive
• Knowledge of CI/CD pipelines (Jenkins, GitHub, Azure DevOps)

Qualifications:

• Bachelors degree in Computer Science, Data Engineering, or a related field
• 6–9 years of relevant experience in data engineering or data infrastructure roles