ETL Data - Senior Engineer
Iris Software · Noida · onsite · Posted 41d ago
Undisclosed5–10 yrsfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
ETL Data - Senior Engineer
Location: Noida, UP, India
Company: Iris Software
Job Description Mandatory Skills: PySpark, Databricks Workflows, Delta Lake on Databricks, Amazon Kinesis
Key Responsibilities - Design scalable data engineering solutions using PySpark and modern distributed data processing frameworks.
- Define data ingestion, transformation, and processing architectures aligned with business and analytical objectives.
- Design and optimize Snowflake or Delta Lake on Databricks solutions to support enterprise-scale data platforms.
- Lead implementation of high-performance batch and streaming data pipelines.
- Define data streaming standards, integration frameworks, and scalable processing patterns.
- Architect workflow orchestration solutions using Apache Airflow or Databricks Workflows.
- Establish monitoring, scheduling, and operational controls for reliable pipeline execution.
- Drive data quality, validation, reconciliation, and governance practices across data engineering solutions.
- Design data engineering solutions following modern Lakehouse architecture principles, data observability practices, and platform engineering standards to improve scalability, reliability, and operational visibility.
- Drive development of business-focused data products by improving data quality, discoverability, usability, documentation, and trusted data consumption across analytical platforms.
- Promote responsible use of AI-assisted engineering capabilities to improve development productivity, testing, documentation, and engineering quality.
- Review data pipeline designs and implementations to ensure adherence to engineering, scalability, and performance standards.
- Troubleshoot complex data processing, workflow, and streaming platform issues through detailed root cause analysis.
- Collaborate with various teams and stakeholders to support end-to-end data platform delivery.
- Demonstrates strong ownership while driving data engineering excellence.
- Collaborate effectively with various teams and business stakeholders to ensure smooth delivery.
- Promotes quality-focused engineering through proactive validation, optimization, and continuous improvement.
- Apply strong analytical thinking to evaluate complex data engineering and platform challenges.
- Demonstrate adaptability while managing evolving technologies, data ecosystems, and business requirements.
- Communicates effectively regarding delivery status, risks, dependencies, and improvement opportunities.
- Maintains high attention to detail across data architecture, pipeline design, testing, and implementation activities.
- Encourages continuous improvement in data engineering practices and platform operations.
- Supports knowledge sharing and mentoring to strengthen team capabilities.
- Balances scalability, performance, reliability, and business priorities while driving delivery excellence.
- Promotes innovation by adopting modern data engineering practices, platform engineering principles, and AI-assisted development approaches to improve engineering productivity and solution quality.
- Big Data - Big Data - PySpark
- Data Science and Machine Learning - Data Science and Machine Learning - Apache Spark
- Data AI - Data Engineering - Data Quality Validation
- Data Science and Machine Learning - Data Science and Machine Learning - Python
- Database - Database Programming - SQL
- Cloud - AWS - AWS SNS, AWS SQS, AWS Kinesis
- Data Science and Machine Learning - Data Science and Machine Learning - Databricks
- Cloud - AWS - Amazon CloudWatch
- Cloud - AWS - AWS S3, S3 glacier, AWS EBS
- Cloud - AWS - Amazon IAM, AWS Secrets Manager, AWS KMS, AWS Cognito
- Cloud - AWS - Amazon API Gateway
- Cloud - AWS - Tensorflow on AWS, AWS Glue, AWS EMR, Amazon Data Pipeline, AWS Redshift
- Cloud - AWS - Dynamo DB, Amazon Aurora, Amazon RDS
- Cloud - AWS - Amazon EC2 / Autoscalaing / Load Balancing, AWS App Runner
- Data Engineering - ETL / Data Integration - ETL Data Integration
- Middleware - API Middleware - API (SOAP, REST)
- Shell - Unix/Linux Shell - Shell
- Communication and collaboration
Sourced from naukri
Skills
Aws GlueAmazon CloudwatchAmazon Ec2AutomationApache SparkAuroraaws kinesisApacheamazon web servicesredshiftAmazon RDSapache airflowAws EmrAnalyticsAWS S3AWS CognitoAPI GatewaysArchitectureBig DataArtificial IntelligenceAws Api Gateway
About Iris Software
Iris Software
IT Services & Consulting
4.3k+
employees
1991
35 years old
Edison, New Jersey
United States (USA)
About
IT company for best tech-solutions to enhance business operations
Industries
IT Services & Consulting
Employee ratings
1,895 reviews
3.8/ 5
Culture
3.7
Career growth
3.3
Work-life
3.8
Employees rate it well for
Work Life Balance381Work Culture267Senior Management254Learning202Salary156
Find them on