HYDERABAD · FULLTIME
AWS Data Engineer
Data Economy
Hyderabad · onsite · Posted 3d ago
Sourced from
Undisclosed4–6 yrsfulltimeHyderabad
Your match
?
Sign in to see your match score, skill gaps & tailored resume.
Section · 01
About this role
Job_Description
Job Title: Data Engineer
Experience Level: 4 - 6 years
Location: Hyderabad / Pune
Experience Level: 4 - 6 years
Location: Hyderabad / Pune
Qualification: BTech/MTech/MCA
Mode of Work: Hybrid
Mode of Work: Hybrid
We are seeking an experienced AWS Data Engineer with deep expertise in Banking, Financial Services, and Insurance systems. The candidate must be familiar at least in one Data ware housing concepts such as ODS/Marts and must have done at least one end-to-end data processing pipeline designing. He/she should be capable of handling the ETL design and implementation solely and would work towards creating a positive and innovation friendly environment. The incumbent would be responsible for creating new capabilities.
Responsibilities
- Develop, maintain, and optimize data processing applications using Python and PySpark.
- Build scalable ETL/ELT pipelines to process large volumes of structured and unstructured data.
- Perform data extraction, cleansing, transformation, aggregation, and validation.
- Work with Apache Spark for distributed data processing and large-scale data transformations.
- Write complex SQL queries and optimize database operations.
- Develop reusable Python modules, functions, and libraries.
- Optimize Spark jobs by analyzing execution plans, partitioning, caching, and performance bottlenecks.
- Work with file formats such as CSV, JSON, Parquet.
- Integrate data from multiple sources, including databases, APIs, and cloud storage.
- Implement error handling, logging, monitoring, and data quality checks.
- Collaborate with data engineers, data architects, analysts, and business stakeholders.
- Follow coding standards, version control practices, and Agile development methodologies.
RequirementsRequirements- Strong programming knowledge in Python, including functions, OOP, exception handling, and data structures.
- Hands-on experience with PySpark DataFrames, Spark SQL, transformations, and actions.
- Good understanding of Apache Spark architecture and distributed computing.
- Strong SQL skills, including joins, sub queries, CTEs, and window functions.
- Experience developing ETL/ELT pipelines and handling large datasets.
- Knowledge of data processing formats such as Parquet, JSON, and CSV.
- Experience with Git and software development best practices.
- Understanding of debugging, performance tuning, and troubleshooting.
Preferred Skills
- Experience with cloud platforms such as AWS, Azure, or Google Cloud.
- Familiarity with Databricks, Azure Data Factory, AWS Glue, or similar data engineering tools.
- Knowledge of data warehouses and lake house architectures.
- Experience with Airflow or other workflow orchestration tools.
- Familiarity with CI/CD pipelines, Docker, and automated testing.
- Knowledge of Delta Lake, Kafka, or streaming data processing.
Benefits- Comprehensive Medical Coverage:
Health insurance of INR 5.0 Lakhs for you and your family (up to 6 members), ensuring complete peace of mind.
- Robust Protection Plans:
Group Personal Accident Insurance and Group Term Life Insurance to safeguard you and your loved ones.
- Retirement Benefits:
-
- Experience with cloud platforms such as AWS, Azure, or Google Cloud.
- Strong programming knowledge in Python, including functions, OOP, exception handling, and data structures.
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
Sourced from naukri
Let the agent run this one for you.
Tailored resume, auto-apply, and referral lookup — in under 2 minutes.
Section · 02
Skills
data qualityApache SparkPerformance Tuningamazon web servicesMonitoringData StructuresJSONcodingPythonData Processing
Section · Company
About Data Economy
Data Economy
IT Services & Consulting
93
employees
2018
8 years old
Dublin, Ohio
United States of America
About
Data Economy is a data and analytics firm, known for its innovative solutions and cost-effective services, delivering value in the global analytical landscape.
Industries
IT Services & Consulting
Employee ratings
79 reviews
3.9/ 5
Culture
3.8
Career growth
3.6
Work-life
3.7
Employees rate it well for
Senior Management12Learning12Career Growth10Work Life Balance8Work Culture7
Find them on