PUNE · FULLTIME
Databricks Engineer
Crisil
Pune · onsite · Posted 40d ago
Sourced from
Undisclosed3–6 yrsfulltimePune
About a minute. You'll make an account on the way through.
agent · working
- Read this posting0.2s
- Found the form on Naukri · 11 questions to answer1.1s
- Decided the answers for all of them3.1s
- Tailor your résumé to this role
- Waiting for you
Section · 01
About this role
Role : Databricks Engineer
Location : Pune
Type of Employment: Permanent
About CRISIL :
We are Indias leading ratings agency. We are also the foremost provider of high-end research to the worlds largest banks and leading corporations. CRISILs majority shareholder is SP Global.
Division : Global Analytical Centre (GAC)
Role Overview:
We are looking for an experienced Databricks Engineer to design, develop, and maintain scalable data engineering solutions using the Databricks Lakehouse platform. The role will involve building robust ETL/ELT pipelines, developing data transformation frameworks, optimizing Spark workloads, and integrating data from multiple enterprise sources.
The ideal candidate should have strong hands on experience with Databricks, Apache Spark, PySpark, SQL, Python, Delta Lake, and cloud data platforms, along with good understanding of data architecture and data governance.
We are looking for an experienced Databricks Engineer to design, develop, and maintain scalable data engineering solutions using the Databricks Lakehouse platform. The role will involve building robust ETL/ELT pipelines, developing data transformation frameworks, optimizing Spark workloads, and integrating data from multiple enterprise sources.
The ideal candidate should have strong hands on experience with Databricks, Apache Spark, PySpark, SQL, Python, Delta Lake, and cloud data platforms, along with good understanding of data architecture and data governance.
Key Responsibilities:
- Design and develop scalable data pipelines using Databricks, Apache Spark and PySpark.
- Build and maintain ETL/ELT pipelines for structured and semi-structured data.
- Develop data transformation and processing workflows using Python, PySpark and SQL.
- Implement and manage Delta Lake tables, including Delta Live Tables / Lakeflow pipelines where applicable.
- Work with Databricks Workflows/jobs for pipeline orchestration and scheduling.
- Integrate data from databases, APIs, files, cloud storage and other enterprise systems.
- Implement data quality, validation, reconciliation and monitoring frameworks.
- Optimize Spark jobs, SQL queries and Databricks workloads for performance and cost.
- Implement appropriate partitioning, caching, clustering and file-management strategies.
- Develop incremental data processing and Change Data Capture (CDC) solutions.
- Work with cloud platforms such as Azure, AWS or GCP.
- Implement security, access controls and data governance in collaboration with platform teams.
- Troubleshoot production data pipelines and resolve performance and data-quality issues.
- Collaborate with Data Scientists, Business Analysts, Application teams and Technology teams.
- Participate in code reviews, technical design discussions and deployment activities.
- Follow Agile development practices and enterprise SDLC standards.
- Maintain technical documentation, data lineage and operational runbooks.
Required Technical Skills:
- Databricks Big Data
- Strong hands-on experience with Databricks
- Apache Spark
- PySpark
Qualification and Essential Skills:
- Bachelor s degree in Computer Science, Software Engineering or a related field.
- Experience as a Python Developer / SQL with a strong portfolio of projects.
- Comprehensive English communication skills.
- Good interpersonal and decision-making skills.
- Ability to work effectively in a team-oriented, global environment with various business partners.
- Well-organized with great attention to detail.
- Ability to multitask and stay poised in a fast-paced, high-pressure environment while also ensuring the highest quality output.
- Flexible, self-starter who is willing to take the initiative and drive tasks to completion with strong execution.
- Must possess strong prioritization and influencing skills, experience in leading highly valued projects.
Eligibility Criteria:
- Not appeared for CRISIL test / interviews in the last six months
- Familiar with Tableau, and PowerBI is required.
- Certification in Databricks, Cloud technologies, AWS, Agile Methodology and Project Management will be added advantage.
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
Sourced from naukri
Section · 02
Skills
CdcAgileApache Sparkagile developmentcloud platformsapisApachecloud technologiesdata architectureclusteringcachingamazon web servicesCloud Storagedata captureCloud DatacloudArchitectureMicrosoft AzureBig Data
Section · Company
About Crisil
Crisil
Analytics & KPO
4.7k+
employees
1987
39 years old
Mumbai,Maharashtra
India
About
Crisil is an agile and innovative, global analytics company driven by its mission of making markets function better.
Industries
Analytics & KPOInternet
Employee ratings
1,936 reviews
3.5/ 5
Culture
3.6
Career growth
2.8
Work-life
3.6
Employees rate it well for
Work Life Balance385Salary355Work Culture280Senior Management209Learning167
Find them on