Data Engineering Consultant - Azure Databricks
Optum Global Solutions · Hyderabad · onsite · Posted 2d ago
Undisclosed8–13 yrsfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
Primary Responsibilities
Required Qualifications
- Design and develop scalable data pipelines using Databricks, Apache Spark, and Python on Azure
- Build cloud-native solutions leveraging Azure Data Lake, Azure Data Factory, and Delta Lake
- Collaborate with Data Science and AI teams to operationalize ML models and embed them into production workflows
- Develop and maintain feature stores, model input pipelines, and real-time/streaming frameworks
- Ensure data quality, governance, and security across the full data lifecycle
- Build reusable frameworks, accelerators, and automation scripts to improve engineering efficiency
- Optimize performance, scalability, and reliability of data workflows and batch/streaming pipelines
- Participate in Agile development processes, including sprint planning, code reviews, and CI/CD pipelines
- Provide production support and on-call coverage, ensuring system stability and rapid issue resolution
- Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements
Required Qualifications
- Bachelor's or Master's degree
- 8+ years of solid experience in SQL / PL-SQL
- 6+ years of hands-on experience with Azure cloud services (ADF, ADLS, Databricks)
- 6+ years of experience in Data Engineering with Python/PySpark
- 6+ years of experience in building ETL/ELT pipelines using Databricks
- 6+ years of experience working in Agile environments
- 2+ years of experience with Databricks Delta Live Tables (DLT)
- Experience with cloud migration (on-prem Databricks/Azure)
- Experience with unit testing, validation, and pipeline testing frameworks
- Familiarity with medallion architecture and SCD2 implementations
- Solid understanding of secure coding practices and vulnerability remediation
- Exposure to GenAI tools (e.g., GitHub Copilot) for engineering productivity
- Proven ability to analyze logs, troubleshoot production issues, and optimize performance
- Demonstrated capability to design and deploy AI-powered solutions responsibly
Sourced from naukri
Skills
AutomationAgileApache SparkCI/CD Pipelinesagile developmentApachecopilotADLSData EngineeringcloudCloud MigrationArchitectureAzure Data FactoryAzure DatabricksAzure Data LakeMicrosoft AzureData PipelineArtificial Intelligence
About Optum Global Solutions
Optum Global Solutions
IT Services & Consulting
33.0k+
employees
2002
24 years old
Hyderabad / Secunderabad, Telangana
India
₹11L PA avg
Avg at Optum Global Solutions
About
We are at the helm of a digital health care revolution, which is being accelerated by a combination of technology and talent. We build and deliver end-to-end digital health solutions for our clients.
Industries
IT Services & ConsultingHardware & NetworkingHealthcareSoftware Product
Employee ratings
10 reviews
4.0/ 5
Culture
3.6
Career growth
3.0
Work-life
3.6
Employees rate it well for
Job Security42Work-Life Balance41
Find them on