Data Engineer - PySpark, Snowflake, AWS & GenAI
D4 Insight · Hyderabad · onsite · Posted today
Undisclosed6–12 yrsfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
Data Engineer - PySpark, Snowflake, AWS GenAI
Job Summary
Key Responsibilities
Required Skills Experience
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
Location: Hyderabad, India
Experience: 6-12 Years
Experience: 6-12 Years
Job Summary
Role Overview: We are looking for experienced Data Engineer with strong expertise in SQL, PySpark, Python, and Snowflake to design and maintain scalable data pipelines and engineering frameworks. The role involves cloud data services, Medallion Architecture, ETL/ELT workflows, and orchestration, with exposure to GenAI technologies as an added advantage.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark, SQL, Python, and Snowflake.
- Build reusable data engineering frameworks for efficient data processing and transformation.
- Implement Medallion Architecture across Bronze, Silver, and Gold layers.
- Develop data ingestion and processing pipelines using AWS Glue and AWS Lambda.
- Implement event-driven and streaming data pipelines using Kafka.
- Orchestrate workflows using Apache Airflow or equivalent tools.
- Optimize SQL queries, Snowflake workloads, and PySpark jobs for performance and scalability.
- Ensure data quality, reliability, and consistency across pipelines.
- Explore GenAI-based data solutions using LLMs, RAG, and LangChain.
Required Skills Experience
- 6-12 years of relevant data engineering experience.
- Strong hands-on expertise in SQL, PySpark, Python, and Snowflake.
- Experience building scalable data pipelines and reusable engineering frameworks.
- Strong understanding of Medallion Architecture and ETL/ELT processes.
- Hands-on experience with Kafka, AWS Glue, and AWS Lambda.
- Experience with Apache Airflow or other workflow orchestration tools.
- Strong problem-solving skills and a focus on data quality and performance.
Good to Have: GCP, additional AWS services, MCP (Model Context Protocol), LLMs, RAG, and LangChain.
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
Sourced from naukri
Skills
data qualityorchestrationApachegoogle cloud platformamazon web servicesworkflowPythonSQLArchitectureData Processing
About D4 Insight
D4 Insight
IT Services & Consulting
125
employees
2015
11 years old
Denver, Colorado
United States
₹23.7L PA avg
Avg at D4 Insight
About
Engaged in decision science, transforming startups into successful enterprises through strategic insights and operational excellence.
Industries
IT Services & Consulting
Employee ratings
10 reviews
3.1/ 5
Culture
2.7
Career growth
3.1
Work-life
2.9
Employees rate it well for
Compensation & Benefits33Overall Company31
Find them on