Databricks

Job DescriptionJoin a collaborative data engineering team where your work directly powers analytics, reporting, and data-driven decisions. In this role, youll help design and build reliable data pipelines on Databricks, transforming raw data into trusted, high-quality datasets that teams can confidently use. Youll partner closely with stakeholders to understand data needs, improve performance, and ensure smooth end-to-end deliveryfrom ingestion to transformation and validation. If you enjoy solving real-world data challenges, optimizing distributed processing, and continuously improving how data platforms operate, this opportunity will help you grow your technical depth while contributing to impactful outcomes. Youll work in an environment that values ownership, learning, and practical innovationwhere clean engineering and teamwork go hand in hand.
Roles ResponsibilitiesKey Responsibilities:

Develop and maintain scalable ETL pipelines using Databricks and PySpark to process large datasets efficiently.

Implement data transformations, cleansing, and enrichment logic aligned to business and analytics requirements.

Optimize Spark jobs for performance and cost by tuning partitions, caching, and cluster configurations where applicable.

Build reusable notebooks/jobs and support scheduling/orchestration of workloads within the Databricks environment.

Perform data validation, reconciliation, and quality checks to ensure accuracy and reliability of curated datasets.

Troubleshoot pipeline failures, analyze logs, and resolve issues to maintain stable production operations.

Collaborate with cross-functional teams to gather requirements, provide estimates, and deliver enhancements iteratively.

Maintain clear technical documentation for pipelines, transformations, and operational runbooks.
Technical Requirement

Primary skills:Technology->Data Engineering->Databricks
Additional ResponsibilityMinimum Qualifications:

Bachelors degree (or equivalent) in Engineering/Technology/Computer Science or related field (BTech/BE/MSc or equivalent).

35 years of experience in data engineering or related roles with hands-on Databricks experience.

Strong hands-on development experience with PySpark for distributed data processing.

Proven experience building and supporting ETL pipelines in production environments.

Ability to analyze data issues, debug Spark/ETL jobs, and implement reliable fixes.
Preferred Qualifications:

Experience designing end-to-end data workflows on Databricks including job scheduling, monitoring, and operational support.

Strong understanding of data modeling concepts and building curated datasets for analytics use cases.

Familiarity with Delta Lake concepts (ACID tables, incremental processing, upserts/merges) and best practices for lakehouse implementations.

Experience with performance tuning techniques for Spark workloads and handling large-scale datasets efficiently.

Exposure to CI/CD practices for data pipelines and maintaining code quality through reviews and standards.
Good to have skills:
Delta Lake, Spark SQL, Data Modeling, Workflow Orchestration, Performance Tuning
Educational RequirementMCA,MSc,MTech,Bachelor of Engineering,BTech
Preferred SkillsTechnology->Data Engineering->Databricks
Service LineData Analytics Unit