SAP, SAP BW

Preferred Qualifications:

• Experience modernizing or integrating SAP BW data with cloud-based lakehouse architectures using Databricks.
• Strong knowledge of Spark optimization techniques (partitioning, caching, joins, file sizing) and production-grade pipeline design.
• Exposure to Delta Lake patterns (ACID tables, MERGE, time travel) and building curated layers for analytics consumption.
• Experience with workflow orchestration and operational monitoring for data pipelines, including alerting and SLA management.
• Proven ability to guide teams through design reviews, code quality improvements, and delivery governance in agile environments.

• Key Responsibilities:

• Lead end-to-end delivery of data engineering solutions leveraging Databricks and PySpark for scalable processing and transformation.
• Design, build, and optimize batch/near-real-time pipelines with strong focus on performance, cost efficiency, and reliability.
• Collaborate with SAP BW stakeholders to understand source models, data semantics, and reporting needs; translate them into modern data platform designs.
• Develop reusable frameworks, coding standards, and best practices for Spark-based development and notebook/job orchestration.
• Ensure data quality through validations, reconciliation checks, and robust error handling/monitoring across pipelines.
• Drive technical discussions, estimate work, plan sprints, and manage delivery risks/dependencies for the team.
• Mentor engineers, conduct code reviews, and enforce secure, maintainable, and well-documented implementations.
• Partner with platform/DevOps teams to support CI/CD, environment management, and production readiness for Databricks workloads.

Minimum Qualifications:

• Bachelors or Masters degree (BTECH, MTECH, MCA, MSC) in Computer Science, Engineering, or a related field.
• 7-9 years of overall experience with strong hands-on expertise in Databricks and PySpark for large-scale data processing.
• Strong experience working with SAP BW concepts and data structures to support analytics/reporting requirements.
• Solid understanding of data modeling, ETL/ELT patterns, and performance tuning in distributed data processing environments.
• Ability to lead technical delivery, collaborate with stakeholders, and communicate clearly across teams.