DevOps Engineer - Oracle Platform

Job Description Python Spark Developer

Position

Python Spark Developer

Experience

3 to 7 Years

Work Location

• Chennai – Sholinganallur SEZ
• Bangalore – RMZ Ecoworld, Bellandur

Job Summary

We are seeking a highly skilled Python Spark Developer with strong expertise in big data technologies and distributed data processing. The ideal candidate should have hands-on experience in Python, Apache Spark, SQL, Unix Shell Scripting, and Airflow, along with a solid understanding of the Hadoop ecosystem.

The candidate will be responsible for designing, developing, optimizing, and maintaining scalable data processing applications while collaborating with cross-functional teams and mentoring junior developers.

Key Responsibilities

• Develop and maintain scalable applications using Python and Apache Spark.
• Design and implement data processing pipelines using:
• Spark Core
• Spark SQL
• Spark Streaming
• Work extensively with the Hadoop Ecosystem for distributed data processing.
• Perform data manipulation, transformation, and analysis using Pandas.
• Develop and manage workflows using Apache Airflow.
• Write efficient and optimized SQL queries and Unix Shell scripts.
• Design low-latency, high-availability, and high-performance applications.
• Perform coding, testing, debugging, troubleshooting, and deployment activities.
• Monitor and optimize application performance, scalability, and usability.
• Implement automation and performance tuning solutions.
• Lead and mentor junior Python developers and provide technical guidance.
• Collaborate with stakeholders, architects, analysts, and development teams to deliver quality solutions.