Data Engineer-Data Platforms-AWS
IBM · Pune · onsite · Posted 1d ago
Undisclosed5–10 yrsfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
Your Role and Responsibilities:
Job Summary
We are seeking an experienced AWS Data Engineer to design, build, and support scalable, secure, and high-performance data platforms on AWS. The role requires strong hands-on experience with AWS services, distributed data processing frameworks, and SQL-driven analytics.
The ideal candidate will work closely with architects, application teams, and business stakeholders to deliver reliable enterprise-grade data solutions.
Key Responsibilities
- Design, develop, and maintain data pipelines and data platforms on AWS
- Build and operate serverless and containerized workloads using AWS Lambda, EC2, and EKS
- Develop batch and large-scale data processing solutions using Apache Spark
- Implement data ingestion, transformation, and analytics using AWS Glue and Athena
- Write complex and high-performance SQL queries for analytics, reporting, and data validation
- Manage and optimize data storage solutions using AWS S3 and RDS
- Design and manage AWS infrastructure using Infrastructure as Code (IaC) with CloudFormation
- Apply AWS security best practices across IAM, VPC networking, data encryption, and access controls
- Develop clean, scalable, and maintainable code using Python and/or Scala
- Monitor, troubleshoot, and optimize performance, cost, and reliability of data workloads
- Collaborate with cross-functional teams to support analytics, reporting, and regulatory use cases
Mandatory Skills & Qualifications
- Strong hands-on experience with AWS services, including o S3, Lambda, EC2, EKS o AWS Glue, Athena o RDS o VPC, IAM o CloudFormation
- Strong SQL skills, including o Writing complex queries, joins, aggregations, and user defined functions o Performance tuning and query optimization
- Good knowledge of AWS Security, including IAM policies, VPC security, encryption, and least-privilege access
- Hands-on programming experience with Python and/or Scala
- Strong experience with Apache Spark for large-scale data processing
- Good communication skills to explain technical concepts to non-technical stakeholders
- Solid understanding of data engineering concepts such as ETL/ELT, data modeling, and distributed systems
Good to Have / Preferred Skills
- Experience with DBT for data transformations
- Hands-on exposure to Snowflake
- Experience working with Databricks
- Familiarity with modern data lake / Lakehouse architectures
- Experience working in enterprise or regulated environments
Nice-to-Have Competencies
- Strong analytical and problem-solving skills
- Ability to work independently and in distributed teams
Required Education:
Bachelor's Degree
Preferred Education:
Master's Degree
Required Technical and Professional Expertise:
- Exposure to AWS Toolset: Experience working with AWS services such as AWS EMR, AWS Glue, Glue Catalog, and Kinesis to design, build, and operate batch and real-time data pipelines.
- Data Pipeline Development: Exposure to developing batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka.
- Data Layer Creation: Experience working with AWS RedShift, Aurora, and DynamoDB to create data layers and migrate data using AWS DMS.
- Open Source Technologies: Exposure to utilizing open source technologies like Apache Airflow and dbt, Spark / Python or Spark / Scala on AWS Platform to support data engineering solutions.
- Data Service Management: Experience scheduling and managing data services on the AWS Platform, ensuring efficient operation of data engineering solutions.
Preferred Technical and Professional Experience:
- Proficiency with AWS Databrew: Experience working with AWS Glue Databrew to support data engineering solutions, including data preparation and data quality tasks.
- Knowledge of Lambda Functions: Exposure to using Lambda functions with Python to support data engineering solutions, including data processing and data transformation tasks.
- Familiarity with RedShift Spectrum: Experience working with RedShift Spectrum to support data engineering solutions, including data warehousing and data analytics tasks.
Years of Experience:5 - 10
Sourced from naukri
Skills
Aws GlueAws LambdaApache SparkApache KafkaAuroraaws kinesisApacheAws Servicesamazon web servicesAWS data engineerredshiftaws cloudformationapache airflowAws EmrAnalyticsAws InfrastructureAWS S3Aws Security
About IBM
IBM
IT Services & Consulting
130k+
employees
1911
115 years old
Bangalore / Bengaluru, Karnataka
India
About
IBM has been dedicated to every client's success and to creating innovations that matter for the world.
Industries
IT Services & ConsultingFinancial ServicesEmerging Technologies
Employee ratings
25,767 reviews
3.9/ 5
Culture
3.9
Career growth
3.0
Work-life
4.0
Employees rate it well for
Work Life Balance4166Salary3460Work Culture2867Learning2246Senior Management1890
Find them on