Data Engineer-Pyspark

Roles and Responsibilities

• Design, develop, test, deploy, and maintain large-scale data pipelines using PySpark.
  
• Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions.
  
• Develop complex ETL processes to extract, transform, and load data from various sources into a centralized warehouse.
  
• Ensure scalability, reliability, and performance of the data pipeline by monitoring its components.
  
• Troubleshoot issues related to data quality, integrity, and security.
  

Desired Candidate Profile

• 6-11 years of experience in Data Engineering with expertise in PySpark.
  
• Strong understanding of Database Development, Data Warehousing Concepts & Tools (e.g., SQL), Big Data Technologies (Hadoop ecosystem).
  
• Experience working on big data projects involving Hive queries writing.
  
• Bachelor's degree (B.Tech / B.E.) in Any Specialization.