Sr Specialist - Architecture

Job Summary

Project Objective of the project is to implement a solution to retain all data documents needed to satisfy a Retention requirements of all Acts and Regulations governing the Credit Cards business eg ECOA FCRA EFTA TILA etc b GS Privacy Policy and c Any litigation and legal holds The solution should enable retrieval of datadocuments to service Regulatory inquiries Audit requests Litigation requests as well as all account documentation requirements from other external parties related to the business

Location Bengaluru Office Full time work from office

Responsibilities

• Design and maintain scalable data pipelines for data ingestion transformation and processing
• Build and optimize ETLELT workflows using PySpark Databricks SQL and Airflow
• Develop data models and solutions for retention reporting and analytics needs
• Ensure data quality integrity and reliability through validation and monitoring frameworks
• Integrate data from multiple sources and optimize processing performance
• Collaborate with crossfunctional teams to deliver data solutions and resolve data issues
• Maintain technical documentation data lineage and mapping artifacts
• Support production deployments troubleshooting and continuous improvement through engineering best practices

Requirements

• PySpark Spark SQL Handson experience writing productiongrade PySpark code
• Databricks Proven experience working within Databricks environments including cluster configuration and Delta Lake
• Apache Airflow Strong experience building and managing pipelines in Apache Airflow
• SQL Mastery Advanced SQL skills window functions CTEs query optimization indexing strategies
• Data Modeling Solid understanding of modern data warehousing concepts
• Version Control Proficiency with Git and CICD deployment workflows
• AWS Experience in Amazon Web Services AWS cloud infrastructure
• ETLELT Proven track record of designing building and optimizing scalable ETLELT pipelines and data platforms

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.