Walk-in || AWS Data Engineer with Pyspark

Job Summary

We are looking for a skilled Data Engineer with strong hands-on experience in Python, PySpark, SQL, AWS Cloud Services, and ETL/Data Engineering. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and data processing solutions in a cloud-based environment. Exposure to GenAI concepts and modern data platforms is highly desirable.

Experience

• Developer: 5-8 years of experience
• Lead Data Engineer: 8-12 years of experience

Key Responsibilities

• Design, develop, and maintain scalable ETL/data processing pipelines using Python and PySpark.
• Build and optimize data ingestion, transformation, and integration workflows.
• Develop and maintain solutions on AWS cloud services such as S3, EMR, Glue, and Airflow.
• Collaborate with business stakeholders, data architects, and development teams to understand data requirements.
• Ensure data quality, reliability, security, and governance across data platforms.
• Optimize SQL queries and improve overall data processing performance.
• Monitor and troubleshoot production data pipelines and workflows.
• Participate in code reviews and follow best practices in data engineering and cloud development.
• Support implementation of AI/GenAI-enabled data solutions where applicable.

Required Skills

• 5+ years of experience in Data Engineering.
• Strong programming skills in Python and PySpark.
• Good experience with SQL and database technologies.
• Hands-on experience with AWS Services:
• Amazon S3
• AWS EMR
• AWS Glue
• Apache Airflow (AWS Managed Airflow knowledge preferred)
• Experience in developing and managing ETL/data integration processes.
• Strong understanding of data warehousing concepts and data modeling.
• Experience with version control tools such as Git.
• Excellent problem-solving and analytical skills.

Preferred Skills

• Basic understanding of Generative AI (GenAI) concepts and use cases.
• Exposure to machine learning or AI-driven data solutions.
• Experience working in Agile/Scrum environments.
• Knowledge of data lake and modern cloud data architectures.

Education

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.

Key Competencies

• Strong communication and stakeholder management skills.
• Ability to work independently and collaboratively in a fast-paced environment.
• Leadership and mentoring capabilities for Lead-level candidates.

Mandatory Skills: Python, PySpark, SQL, AWS (S3, EMR, Glue), Airflow, ETL/Data Engineering

Good to Have: GenAI basics, Data Modeling, Data Warehousing, Agile methodology.