PySpark Data Engineer

Role & Responsibilities

• Design and develop scalable data pipelines using PySpark.
• Build ETL workflows and data transformation solutions.
• Process large-scale structured and unstructured datasets.
• Develop and optimize Spark applications for performance.
• Work with SQL and Spark SQL for data analysis and processing.
• Ensure data quality, scalability, and reliability.
• Troubleshoot and resolve data pipeline issues.
• Collaborate with business and technical teams.

Preferred Candidate Profile

• 2-5 years of experience in Data Engineering.
• Strong hands-on experience with PySpark and Apache Spark.
• Strong Python programming skills.
• Good SQL knowledge.
• Experience in ETL and Big Data technologies.
• Exposure to Databricks, Hadoop, or Hive is preferred.
• Strong analytical and problem-solving abilities.