Data Engineer

Project Role : Data Engineer

Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.

Must have skills : Databricks Unified Data Analytics Platform

Good to have skills : Business Intelligence (BI) Reporting Tools, Python (Programming Language), Data Engineering

Minimum 2 Year(s) Of Experience Is Required

Educational Qualification : 15 years full time education

Summary:

As a Data Engineer, a typical day involves designing, developing, and maintaining comprehensive data solutions that support the generation, collection, and processing of data. This role includes creating efficient data pipelines to facilitate smooth data flow, ensuring the accuracy and reliability of data, and implementing processes to extract, transform, and load data across various systems. The position requires continuous monitoring and optimization of data workflows to support organizational needs and enable seamless data migration and deployment.

Data Engineering Developer

Roles & Responsibilities:

• Build and maintain ingestion pipelines from source systems into Azure Data Lake develop transformation logic using Databricks (notebooks, Delta Live Tables), Python and PySpark implement incremental, change-data and full-refresh patterns build and manage ADF orchestration workflows implement data-quality checks and validation via Unity Catalog maintain data lineage and metadata write unit and integration tests support onboarding of new sources provide L2/L3 support for production pipelines.
  
  

Professional & Technical Skills:

• Must To Have Skills: Proficiency in Databricks Unified Data Analytics Platform.
• Good To Have Skills: Experience with Python (Programming Language), Data Engineering, Business Intelligence (BI) Reporting Tools.
• Strong understanding of data pipeline architecture and ETL processes.
• Experience with data quality assurance and validation techniques.
• Familiarity with cloud-based data platforms and distributed computing.
• Ability to optimize data workflows for performance and scalability.
  
  

Additional Information:

• The candidate should have minimum 2 years of experience in Databricks Unified Data Analytics Platform.
• This position is based at our Bengaluru office.
• A 15 years full time education is required.