SENIOR DATA ENGINEER - Databricks

Senior Data Engineer (Databricks)

Mandatory Skills

Spark,Databricks Engineer,Kafka,Stream Processing,SQL Development,Python,Snowflake

Skill to Evaluate

Spark,Databricks Engineer,Snowflake SQL,Kafka,Stream Processing,SQL Development,Python,Snowflake

Experience

6 to 8 Years

Location

Bengaluru

Job Description

Architecture & Strategy: End-to-end design of scalable, secure, and highly available data architectures leveraging modern cloud data ecosystems (Databricks?and?Snowflake).

Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable streaming and batch data pipelines (ETL/ELT) to process complex, large-scale datasets.

Cloud Architecture: Architect and deploy scalable enterprise data platform components natively within the?AWS ecosystem, ensuring tight integration with core security, IAM, and networking protocols.

API Ingestion & Orchestration:?Design and implement robust data ingestion frameworks leveraging?Databricks APIs?and external REST/GraphQL APIs for automated workflows, platform orchestration, and data delivery.

Real-time Processing:?Design and implement robust frameworks for real-time data ingestion and processing to solve business-critical, low-latency use cases.

Hybrid Data Modelling:?Harmonize "old school" relational data warehousing patterns (Kimball/Inmon, Star/Snowflake schemas) with unstructured/semi-structured modern paradigms.

Technical Leadership:?Act as a core problem-solver for complex data bottlenecks, provide technical governance, and mentor engineering teams on data best practices.

AI Integration:?Collaborate with Data Science and AI teams to architect data layers that seamlessly support LLMs, Machine Learning pipelines, and advanced analytics solutions.

Education Qualificaiton

B Tech/ M Tech

Job Title

Senior Data Engineer (Databricks)

Roles & Responsibilities

Architecture & Strategy: End-to-end design of scalable, secure, and highly available data architectures leveraging modern cloud data ecosystems (Databricks?and?Snowflake).

Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable streaming and batch data pipelines (ETL/ELT) to process complex, large-scale datasets.

Cloud Architecture: Architect and deploy scalable enterprise data platform components natively within the?AWS ecosystem, ensuring tight integration with core security, IAM, and networking protocols.

API Ingestion & Orchestration:?Design and implement robust data ingestion frameworks leveraging?Databricks APIs?and external REST/GraphQL APIs for automated workflows, platform orchestration, and data delivery.

Real-time Processing:?Design and implement robust frameworks for real-time data ingestion and processing to solve business-critical, low-latency use cases.

Hybrid Data Modelling:?Harmonize "old school" relational data warehousing patterns (Kimball/Inmon, Star/Snowflake schemas) with unstructured/semi-structured modern paradigms.

AI Integration:?Collaborate with Data Science and AI teams to architect data layers that seamlessly support LLMs, Machine Learning pipelines, and advanced analytics solutions.

Type of Employment

Contract

Project Details

Architecture & Strategy: End-to-end design of scalable, secure, and highly available data architectures leveraging modern cloud data ecosystems (Databricks?and?Snowflake).

Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable streaming and batch data pipelines (ETL/ELT) to process complex, large-scale datasets.

Cloud Architecture: Architect and deploy scalable enterprise data platform components natively within the?AWS ecosystem, ensuring tight integration with core security, IAM, and networking protocols.

API Ingestion & Orchestration:?Design and implement robust data ingestion frameworks leveraging?Databricks APIs?and external REST/GraphQL APIs for automated workflows, platform orchestration, and data delivery.

Real-time Processing:?Design and implement robust frameworks for real-time data ingestion and processing to solve business-critical, low-latency use cases.

Hybrid Data Modelling:?Harmonize "old school" relational data warehousing patterns (Kimball/Inmon, Star/Snowflake schemas) with unstructured/semi-structured modern paradigms.

Technical Leadership:?Act as a core problem-solver for complex data bottlenecks, provide technical governance, and mentor engineering teams on data best practices.

AI Integration:?Collaborate with Data Science and AI teams to architect data layers that seamlessly support LLMs, Machine Learning pipelines, and advanced analytics solutions.

Project Duration

12

Shift Timings

General