Senior PySpark Developer

We are looking for an experienced Senior PySpark Developer with 8+ years of experience in Data Engineering and Big Data technologies. The ideal candidate should possess strong expertise in PySpark, Python, Spark SQL, ETL development, Data Warehousing, and Cloud platforms (AWS/Azure/GCP). The candidate will be responsible for designing, developing, and optimizing large-scale data processing solutions.

• Design and develop scalable data pipelines using PySpark and Apache Spark.
• Build and optimize ETL/ELT frameworks for processing large volumes of structured and unstructured data.
• Develop high-performance Spark applications using Spark SQL, DataFrames, and RDDs.
• Work closely with data architects and business teams to define data requirements.
• Implement data quality, validation, and governance processes.
• Optimize Spark jobs for performance, memory management, and scalability.
• Integrate data from various sources including databases, APIs, cloud storage, and streaming platforms.
• Support production deployments and resolve performance bottlenecks.
• Mentor junior developers and participate in code reviews.

• 8+ years of IT experience.
• Strong hands-on experience in PySpark and Python.
• Expertise in Apache Spark, Spark SQL, DataFrames, and ETL Development.
• Strong SQL and database concepts.
• Experience with Hadoop Ecosystem (Hive, HDFS, YARN).
• Experience in Data Warehousing and Data Modeling.
• Hands-on experience with Unix/Linux.

• MTech
• Bachelor of Engineering
• BTech

• Big Data - Data Processing - PySpark

• Data Analytics Unit