Data Engineering Consultant - ADB, ADF, Azure, Spark and SQL
Optum Global Solutions · Bengaluru · onsite · Posted 2d ago
Undisclosed7–12 yrsfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
Responsibilities
- Design and Build Data Infrastructure: Develop and maintain scalable data pipelines (ETL/ELT) for large-scale data processing
- Data Architecture: Lead architectural decisions for data warehousing and big data solutions
- Optimize Performance: Improve database performance, query efficiency, and data workflows
- Data Quality and Governance: Implement frameworks for data validation, monitoring, and compliance
- Collaboration: Work with data scientists, analysts, and business teams to enable analytics and machine learning
- Mentorship: Guide junior engineers and promote best practices in data engineering
- Automation and Deployment: Create automated testing and deployment processes for data systems
- Troubleshooting: Resolve complex data issues and ensure system reliability
- Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
- Undergraduate degree or equivalent experience
- 7+ years of experience in Spark/Scala, Azure Databricks, Azure Data Factory, Airflow and Kafka. Proficient in coding, testing, implementing, debugging, and documenting the complex Spark Jobs
- Programming: Solid in Azure Databricks, Python/Scala, SQL experience
- ETL Tools and Big Data: Hands-on experience with Spark, Hadoop, or similar frameworks
- Cloud Platforms: Azure, (or GCP) experience
- Containerization and Orchestration: Docker, Kubernetes
- Data Modeling and Warehousing: Expertise in relational and NoSQL databases
- Distributed Systems: Knowledge of data partitioning and parallel processing
- Data Security and Compliance: Familiarity with GDPR, HIPAA, or similar standards
- Experience with Snowflake
- Streaming Technologies: Kafka or similar message queuing systems
- AI and Machine Learning Deployment: implementing AI solutions (RAG development, Agents creation), Exposure to ML model integration
- CI/CD for Data: Jenkins or other DevOps tools for data workflows
- BI Tools: Tableau, Power BI for reporting
Sourced from naukri
Skills
AutomationDockercloud platformsbi toolsautomated testingdata architectureapache airflowAnalyticsData EngineeringcloudArchitectureAzure Data FactoryAzure DatabrickscomplianceAi SolutionsMicrosoft AzureBig DataArtificial Intelligence
About Optum Global Solutions
Optum Global Solutions
IT Services & Consulting
33.0k+
employees
2002
24 years old
Hyderabad / Secunderabad, Telangana
India
₹11L PA avg
Avg at Optum Global Solutions
About
We are at the helm of a digital health care revolution, which is being accelerated by a combination of technology and talent. We build and deliver end-to-end digital health solutions for our clients.
Industries
IT Services & ConsultingHardware & NetworkingHealthcareSoftware Product
Employee ratings
10 reviews
4.0/ 5
Culture
3.6
Career growth
3.0
Work-life
3.6
Employees rate it well for
Job Security42Work-Life Balance41
Find them on