Data Engineer

Project Role :Data Engineer


Project Role Description :Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.
Must have skills :Databricks Unified Data Analytics Platform

Good to have skills :Business Intelligence (BI) Reporting Tools, Python (Programming Language), Data Engineering
Minimum 7.5 year(s) of experience is required

Educational Qualification :15 years full time education

Summary:
As a Data Engineer, a typical day involves designing, developing, and maintaining comprehensive data solutions that support the generation, collection, and processing of data. The role includes creating efficient data pipelines to facilitate smooth data flow, ensuring the accuracy and integrity of data throughout its lifecycle, and implementing processes to extract, transform, and load data across various systems. This position requires continuous collaboration with different teams to optimize data handling and support organizational data needs effectively.

Data Engineering Lead

Roles Responsibilities:
- Build and maintain ingestion pipelines from source systems into Azure Data Lake develop transformation logic using Databricks (notebooks, Delta Live Tables), Python and PySpark implement incremental, change-data and full-refresh patterns build and manage ADF orchestration workflows implement data-quality checks and validation via Unity Catalog maintain data lineage and metadata write unit and integration tests support onboarding of new sources provide L2/L3 support for production pipelines

Professional Technical Skills:
-
Must To Have Skills:
Proficiency in Databricks Unified Data Analytics Platform, Data Engineering, Business Intelligence (BI) Reporting Tools, Python (Programming Language).
- Good To Have Skills:Data Engineering, Business Intelligence (BI) Reporting Tools, Python (Programming Language).
- Strong knowledge of building and managing scalable data pipelines and workflows.
- Experience with data integration, transformation, and migration techniques.
- Familiarity with cloud-based data platforms and distributed computing environments.
- Ability to troubleshoot and resolve complex data-related issues efficiently.

Required skills:
Azure Data Factory, Databricks, Python, Spark

- The candidate should have minimum 7.5 years of experience in Databricks Unified Data Analytics Platform.
- This position is based at our Bengaluru office.
- A 15 years full time education is required.


Qualification
15 years full time education