Data Engineer (Integration & Automation)

Job Description Data Engineer

Job Title: Data Engineer
Experience: 4–6 Years
Employment Type: Full-Time
Department: IT Services

Location: Vadodara, Corporate Office.

Job Summary

We are looking for an experienced Data Engineer to join our IT team. The candidate will be responsible for developing, implementing, maintaining, and supporting solutions across ETL/Data Integration, Cloud Data Lake/Data Warehouse, and Data Analytics, Robotic Process Automation (RPA).

The role requires hands-on experience with Informatica/IICS, SQL, and Google BigQuery, Automation Anywhere / Automation 360,. The candidate will work with business and technical teams to understand requirements, design scalable solutions, develop and deploy automations and data pipelines, and provide production support.

Key Responsibilities:

1. Informatica / ETL & Data Integration

• Design, develop, test, deploy, and maintain ETL workflows using Informatica / Informatica Intelligent Cloud Services (IICS).
• Develop data integration pipelines from multiple source systems including:
• SAP
• Oracle
• SQL databases
• Excel/CSV files
• APIs
• Cloud data sources
• Analyze source data and develop appropriate source-to-target mappings.
• Perform data extraction, transformation, cleansing, validation, enrichment, and loading.
• Implement full load and incremental/delta load strategies.
• Design ETL processes for historical data and change data processing.
• Implement data quality checks, reconciliation, and validation mechanisms.
• Develop reusable mappings, mapping tasks, workflows, and parameterized ETL components.
• Implement error handling, logging, restart/recovery mechanisms, and operational monitoring.
• Monitor production ETL jobs and troubleshoot failures.
• Analyze ETL performance and optimize workflows for improved processing time and resource utilization.
• Work with technical and business teams to resolve data-related issues.
• Perform impact analysis for changes to source systems, mappings, and target structures.
• Maintain technical documentation, source-to-target mapping documents, and operational procedures.
• Participate in code reviews and ensure development follows ETL standards and best practices.

2. Google BigQuery & Data Lake / Data Warehouse

• Design and develop data processing solutions using Google BigQuery.
• Develop complex SQL queries for data extraction, transformation, aggregation, validation, and analytics.
• Create and maintain BigQuery datasets, tables, views, materialized views, and stored procedures where required.
• Design data models suitable for analytical and reporting requirements.
• Develop and maintain cloud-based Data Lake / Data Warehouse solutions.
• Work with structured and semi-structured data from SAP, databases, APIs, and files.
• Implement efficient data ingestion and transformation pipelines.
• Design and implement incremental data loading and historical data management strategies.
• Implement BigQuery partitioning and clustering based on data access patterns.
• Optimize SQL queries and BigQuery workloads for performance and cost efficiency.
• Monitor data processing and investigate query/data pipeline performance issues.
• Implement data quality checks and source-to-target reconciliation.
• Work with large datasets and optimize data processing approaches.
• Support data migration and data transformation initiatives.
• Assist in designing scalable and maintainable cloud data architectures.
• Collaborate with BI/analytics teams to provide trusted datasets for reporting and dashboards.

3. Integration & Automation

• Develop end-to-end solutions combining RPA, ETL, SAP, databases, APIs, and BigQuery.
• Identify opportunities to integrate RPA automation with data engineering workflows.
• Automate manual data extraction, transformation, validation, and reporting processes.
• Develop solutions for transferring data between enterprise applications and cloud platforms.
• Work with APIs and web services for system integration.
• Handle JSON, XML, CSV, Excel, and other data formats.
• Design reliable and reusable integration components.
• Troubleshoot issues across the complete automation and data pipeline.

Required Technical Skills

Informatica / ETL

• 3+ years of practical experience with Informatica / Informatica IICS.
• Strong understanding of ETL/ELT concepts.
• Experience developing Informatica mappings, mapping tasks, and workflows.
• Experience with source-to-target mapping and data transformation.
• Experience with full and incremental data loads.
• Experience integrating data from multiple source systems.
• Strong understanding of data cleansing, validation, reconciliation, and error handling.
• Experience troubleshooting and optimizing ETL workflows.

SQL / BigQuery

• Strong SQL development skills.
• Hands-on experience with Google BigQuery.
• Understanding of BigQuery datasets, tables, views, and schemas.
• Experience with partitioning and clustering.
• Experience optimizing SQL queries and BigQuery processing costs.
• Understanding of data warehouse concepts and dimensional modeling is preferred.

Programming / Scripting

Good working knowledge of at least one programming/scripting language:

• Python /Java /JavaScript

Python experience for data processing, automation, or API integration will be an advantage.

Experience

4–6 Years

The candidate should have demonstrated hands-on experience in one or more of the following areas, with the ability to work across the complete technology stack:

• Automation Anywhere / Automation 360
• Informatica / IICS
• SQL
• Google BigQuery
• Data Lake / Data Warehouse
• ETL / Data Integration
• Python / scripting
• SAP automation or SAP data integration

Candidates with strong experience across RPA + Informatica + BigQuery will be preferred.