Pyspark Developer

We are looking for an experienced PySpark/Spark Developer with 5-8 years of experience in designing, developing, and optimizing big data solutions. The ideal candidate should possess strong expertise in PySpark, Apache Spark, SQL, and distributed data processing technologies.

The candidate will work closely with business stakeholders, architects, and development teams to build scalable and high-performance data engineering solutions.

Role & Responsibilities

• Design, develop, and maintain scalable data processing solutions using PySpark and Apache Spark.
• Build and optimize ETL pipelines for processing large datasets.
• Develop data transformation workflows using Spark SQL and Python.
• Work with Hadoop ecosystem technologies including Hive and HDFS.
• Optimize Spark jobs for performance, scalability, and reliability.
• Develop and support batch and large-scale distributed processing applications.
• Collaborate with business stakeholders and technical teams to understand requirements.
• Troubleshoot and resolve data processing and performance issues.
• Ensure data quality, governance, and operational excellence.
• Participate in Agile ceremonies and project delivery activities.

Preferred Candidate Profile

• 5-8 years of experience in Data Engineering and Big Data technologies.
• Strong hands-on experience in PySpark and Apache Spark.
• Excellent Python and SQL skills.
• Experience with Spark SQL, ETL development, and Data Warehousing.
• Experience working with large-scale distributed data platforms.
• Exposure to Databricks, Hadoop, Hive, or Spark Streaming is preferred.
• Knowledge of AWS or Azure cloud platforms is an added advantage.
• Strong analytical and problem-solving skills.
• Good communication and stakeholder management abilities.