Data Engineer

Key Responsibilities

• Design, develop, and maintain data pipelines using Databricks and Apache Spark.
• Integrate and manage Composite applications for enterprise data workflows.
• Optimize ETL/ELT processes for performance, scalability, and reliability.
• Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality datasets.
• Implement best practices for data governance, security, and compliance.
• Monitor and troubleshoot data workflows to ensure high availability and accuracy.
• Contribute to architectural decisions for cloud based big data solutions
• Hands-on experience with Databricks (clusters, notebooks, Delta Lake, MLflow).
• Knowledge of Composite applications for enterprise integration.
• Solid understanding of data warehousing, ETL/ELT pipelines, and distributed systems.
• Experience with cloud platforms (Azure, AWS or GCP).
• Familiarity with SQL, Python, and REST APIs.
• Strong problem-solving and debugging skills in large-scale data environments.