Sr. Developer

Job Summary Sr Developer role in a global organization focusing on data engineering solutions for life and annuities insurance using Databricks SQL Databricks Workflows and PySpark. The role involves building scalable data pipelines optimizing analytical workloads and ensuring high quality data products in a hybrid work model with day shift and no travel.

Responsibilities

• Design robust data pipelines in Databricks using Databricks SQL and PySpark to transform complex life and annuities insurance data into reliable analytics ready datasets
• Develop reusable PySpark modules that standardize ingestion cleansing and enrichment of policy claims and actuarial data while meeting performance expectations and coding standards
• Configure and manage Databricks Workflows to orchestrate end to end batch and near real time data processing jobs supporting reporting and analytics needs for insurance stakeholders
• Optimize Databricks SQL queries and PySpark jobs by tuning partitions caching strategies and cluster configurations to achieve efficient resource usage and faster insights
• Implement data quality checks validation rules and reconciliation processes that improve the accuracy and completeness of life and annuities insurance datasets used across business functions
• Collaborate with business analysts and actuaries to translate insurance product rules policy events and financial calculations into clear and maintainable data engineering logic
• Document technical designs job dependencies transformation logic and operational procedures to ensure transparency maintainability and smooth knowledge transfer within the team
• Partner with testing and operations teams to define test cases support defect resolution and enable stable deployment of Databricks solutions into production environments
• Monitor production workflows in Databricks investigate failures or performance regressions and implement corrective actions that protect data reliability and service commitments
• Apply secure coding practices implement access controls and follow compliance guidelines relevant to financial and insurance data handling across all data engineering activities
• Contribute to continuous improvement by identifying automation opportunities refactoring legacy code and adopting emerging Databricks features that deliver better outcomes for the company
• Support business reporting and advanced analytics initiatives by exposing curated data models reusable views and standardized metrics for life and annuities insurance use cases
• Work effectively in a hybrid environment by coordinating with on site and remote team members ensuring timely communication and consistent progress on shared deliverables

Qualifications

• Possess professional experience between six and eight years in data engineering or related roles with a strong focus on large scale analytics solutions
• Demonstrate deep proficiency in Databricks SQL including development of complex queries views and performance optimized data models for analytical consumption
• Show hands on expertise in PySpark programming including data frame operations user defined functions and performance tuning techniques on distributed datasets
• Exhibit practical experience configuring and managing Databricks Workflows including job scheduling dependency handling and monitoring of production pipelines
• Bring solid domain knowledge in life and annuities insurance including familiarity with policies riders premiums benefits and related data structures
• Display strong understanding of data warehousing concepts dimensional modeling and best practices for building curated layers supporting reporting and analytics
• Apply sound knowledge of software engineering practices such as version control code review and modular design to maintain high quality data engineering solutions
• Communicate effectively with cross functional partners by explaining technical designs in business friendly language and incorporating feedback into implementation plans
• Adhere to organizational standards for security compliance and documentation while contributing ideas that enhance efficiency and long term maintainability of solutions

Certifications Required Preferred certifications include Databricks Data Engineer Associate or Professional and cloud platform certifications relevant to data engineering