BENGALURU · FULLTIME
Data Platform Modernization Engineer - Junior (Databricks)
NTT DATA BUSINESS SOLUTIONS
Bengaluru · onsite · Posted 47d ago
Sourced from
Undisclosed6–11 yrsfulltimeBengaluru
About a minute. You'll make an account on the way through.
agent · working
- Read this posting0.6s
- Found the form on Naukri · 9 questions to answer1.0s
- Decided the answers for all of them2.7s
- Tailor your résumé to this role
- Waiting for you
Section · 01
About this role
Req ID: phone_number
Job Summary We are currently seeking a Data Platform Modernization Engineer - Junior (Databricks) to join our team in Bangalore, Karntaka, India.
Day-to-Day Job Duties - Analyze existing Informatica workflows, mappings and AWS Glue jobs to understand source-to-target mappings and transformation logic.
- Develop Databricks notebooks and data pipelines to migrate existing ETL workloads to the Databricks Lakehouse.
- Develop data transformations using Python, PySpark, Spark SQL and Databricks SQL.
- Implement Bronze, Silver and Gold layer processing following established project architecture and coding patterns.
- Develop batch and incremental processing pipelines using Delta Lake and established watermark/control mechanisms.
- Develop and maintain Databricks Jobs/Workflows and Lakeflow Declarative Pipelines as required.
- Reuse common project frameworks for logging, auditing, data quality, error handling and notifications.
- Convert Informatica transformations, mappings, lookups and business rules into equivalent Databricks implementations while preserving existing business logic.
- Support migration of AWS Glue jobs and database-based processing into Databricks.
- Perform unit testing and source-to-target data reconciliation for migrated pipelines.
- Investigate and resolve data, SQL, PySpark and pipeline execution issues.
- Support integration testing, regression testing, UAT and production validation.
- Work with senior developers and technical leads to resolve technical issues and dependencies.
- Follow established coding, performance, security and development standards.
- Participate in code reviews and incorporate review feedback.
- Prepare technical documentation and support knowledge transfer for migrated workloads.
- Support deployment, monitoring, production stabilization and cutover activities.
- 36 years of experience in data engineering, ETL development or application/data platform development.
- Hands-on experience with Databricks and Apache Spark.
- Strong development skills in Python/PySpark and SQL.
- Experience with:
- Databricks notebooks
- Spark SQL / Databricks SQL
- Delta Lake
- Databricks Jobs/Workflows
- Experience developing batch or incremental ETL/ELT pipelines.
- Experience with one or more ETL technologies such as Informatica PowerCenter, AWS Glue, SSIS or similar.
- Experience working with relational databases and SQL-based data processing.
- Understanding of data transformation, source-to-target mapping and ETL development.
- Experience with unit testing, data validation and troubleshooting.
- Basic understanding of Bronze/Silver/Gold or other layered data architectures.
- Ability to work within an established development framework and follow coding standards.
- Good analytical, problem-solving and communication skills.
- Bachelor's degree in Computer Science, Information Technology, Engineering or equivalent experience.
- Databricks certification.
- Experience with Delta Live Tables.
- Experience with Unity Catalog.
- Experience with Databricks Asset Bundles and CI/CD.
- Experience with AWS S3, Glue and Redshift.
- Experience with Auto Loader or streaming ingestion.
- Experience with CDC and SCD Type 1 / Type 2 processing.
- Experience with data quality, audit and error-handling frameworks.
- Experience with Informatica PowerCenter mappings/workflows.
- Experience with AWS services including S3, Glue, Lambda and Redshift.
- Experience with Spark/Delta performance optimization.
- Experience working in a cloud data modernization or migration project.
Sourced from naukri
Section · 02
Skills
data qualityAws GlueCdcApache SparkAuditingcoding standardsApacheAws Servicesamazon web servicesClient SuccessBankingAWS S3Cloud DataData EngineeringcloudCommunication SkillsArchitectureData ProcessingData PipelineArtificial Intelligence
Section · Company
About NTT DATA Business Solutions
NTT DATA BUSINESS SOLUTIONS
IT Services & Consulting
4k+
employees
1989
37 years old
Hyderabad / Secunderabad, Telangana
India
About
NTT DATA Business Solutions helps clients innovate and transform through expert SAP solutions, advisory, implementation, and managed services.
Industries
IT Services & Consulting
Employee ratings
2,105 reviews
4.1/ 5
Culture
4.0
Career growth
3.5
Work-life
4.0
Employees rate it well for
Work Life Balance176Salary146Senior Management116Work Culture99Work Environment83
Find them on