Big Data Developer

Job Description:

• Design, develop, and maintain data pipelines and ETL workflows using Databricks, Spark, and ADF.
• Implement data lakehouse architecture and optimize data storage and retrieval for performance and cost efficiency.
• Write efficient, reusable, and scalable Python code for data processing and automation tasks.
• Orchestrate complex workflows using Azure Data Factory and integrate with other Azure services.
• Develop and maintain Power BI dashboards and reports, ensuring accurate and timely insights for stakeholders.
• Ensure data quality, security, and governance across all data solutions.
• Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data products.
• Monitor and troubleshoot data pipelines, ensuring high availability and reliability.
  Stay updated on emerging technologies and best practices in cloud data engineering.
• Experience with data lakehouse architecture, Delta Lake, and data modeling.
  Knowledge of CI/CD pipelines, DevOps practices, and version control (Git).