Databricks, PySpark, Python - Data Engineer

Responsibilities :

Lead the design, development, and deployment of data engineering solutions using Databricks and PySpark. Develop scalable ETL/ELT pipelines and data transformation frameworks. Write efficient Python and SQL code for data processing and analytics. Work closely with stakeholders to understand business requirements and translate them into technical solutions. Optimize Spark jobs for performance, scalability, and cost efficiency. Ensure data quality, governance, and security standards are followed. Review code, mentor team members, and provide technical guidance. Support production deployments, incident troubleshooting, and root cause analysis. Participate in Agile ceremonies and drive sprint deliverables.

Technical and Professional Requirements:

7-9 years of overall IT experience. Strong hands-on expertise in Databricks and PySpark. Proficiency in Python programming and SQL. Experience building enterprise-scale data pipelines and data lake solutions. Strong understanding of distributed computing concepts and Spark architecture. Experience in performance tuning and optimization. Excellent stakeholder management and team coordination skills.