Data Engineer

Miles Education has partnered with a leading global professional services firm (our client) to hire a Data Engineer. We are seeking experienced professionals with strong expertise in PySpark, Microsoft Fabric, and SQL to design, build, and optimize scalable data solutions that support business intelligence, analytics, and enterprise reporting initiatives.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark.
• Build and manage data solutions within Microsoft Fabric, including data ingestion, transformation, and orchestration.
• Develop and optimize complex SQL queries, stored procedures, and database objects.
• Integrate data from multiple source systems and ensure data quality, consistency, and reliability.
• Implement ETL/ELT processes for structured and unstructured data.
• Monitor and troubleshoot data workflows, identifying performance bottlenecks and implementing improvements.
• Collaborate with business stakeholders, analysts, and engineering teams to understand data requirements.
• Support data warehousing, reporting, and analytics initiatives.
• Ensure adherence to data governance, security, and best practices.

Required Skills (Must Have)

• 5 to 12 years of experience in Data Engineering.
• Strong hands-on experience with PySpark.
• Extensive experience with Microsoft Fabric.
• Advanced proficiency in SQL and database optimization.
• Experience building and maintaining ETL/ELT pipelines.
• Strong understanding of data modeling, data warehousing, and large-scale data processing.
• Experience working with cloud-based data platforms.
• Strong analytical and problem-solving skills.

Preferred Skills

• Experience with Azure Data Services.
• Knowledge of Data Lakes, Lakehouse architecture, and Delta tables.
• Exposure to Power BI and reporting solutions.
• Understanding of DevOps, CI/CD, and version control tools.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field.
• Strong communication and stakeholder management skills.

What We're Looking For

• Self-driven and proactive professional.
• Ability to work in a fast-paced, collaborative environment.
• Strong focus on scalability, performance, and data quality.
• Excellent troubleshooting and problem-solving capabilities.
  Notice Preference: Immediate to 30 Days Preferred