Azure Databricks-Data Engineer

Position: Data Bricks-Data Engineer

Experience: 2 to 5 years

Location: Hyderabad

Employment Type: Full-Time

Roles & Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks.
• Build and optimize ETL/ELT processes for large-scale data ingestion and transformation.
• Develop Spark and PySpark applications for batch and real-time processing.
• Implement Lakehouse architecture using Delta Lake.
• Create and maintain data models, tables, and data warehouses.
• Optimize Databricks jobs, Spark performance, and query execution.
• Integrate data from multiple cloud and on-premises sources.
• Ensure data quality, governance, and security standards.
• Work with Unity Catalog for data governance and access management.
• Automate deployments using CI/CD and Infrastructure as Code (Terraform).
• Monitor and troubleshoot production data pipelines.
• Collaborate with Data Scientists, BI Developers, and Business Stakeholders.
• Support cloud migration and data modernization initiatives.