Technical Lead PySpark / Pune /

Technical Lead – PySpark

Location: Pune

Experience: 5–8 Years

Primary Skill: PySpark

Job Description

We are looking for a Technical Lead – PySpark with strong hands-on experience in building and optimizing scalable data processing solutions. The ideal candidate should have solid expertise in PySpark, Python, SQL, and data engineering, along with the ability to provide technical guidance to the development team.

Key Responsibilities

• Design, develop, and maintain scalable data processing solutions using PySpark.
• Build and optimize ETL/data pipelines for large datasets.
• Perform performance tuning and optimization of Spark/PySpark jobs.
• Troubleshoot complex data processing and application issues.
• Conduct code reviews and ensure development best practices.
• Provide technical guidance and mentoring to team members.
• Collaborate with cross-functional teams to translate requirements into technical solutions.
• Ensure solutions are scalable, reliable, and maintainable.

Required Skills

• Strong hands-on development experience in PySpark.
• Strong programming skills in Python.
• Good understanding of Apache Spark and distributed data processing.
• Strong knowledge of SQL and data transformations.
• Hands-on experience developing ETL/data pipelines.
• Experience in Spark/PySpark performance optimization.
• Good problem-solving and troubleshooting skills.
• Experience in technical leadership, code reviews, and team guidance.

Preferred Skills

• Exposure to AWS, Azure, or GCP data platforms.
• Understanding of big-data and modern data-engineering ecosystems.
• Familiarity with Git, CI/CD, and software development best practices.