SAP Datasphere & Databricks

Key Responsibilities:

• Hybrid Data Architecture: Design and implement a seamless data mesh/lakehouse architecture integrating SAP Datasphere (Data & Business Layers and Data Spaces.) with the Databricks Lakehouse Platform.
• Data Pipeline Engineering: Develop and maintain scalable, low-latency ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables (DLT), and Datasphere Replication/Transformation flows.
• Semantic Layer Integration: Expose SAP Datasphere analytical datasets and business models seamlessly to Databricks, ensuring business logic (like hierarchies and currencies) is preserved or effectively translated.
• Federation vs. Ingestion Strategy: Establish governance on when to federate queries live from Databricks to Datasphere versus when to ingest and persist heavy SAP datasets into Delta Lake.
• Unified Governance: Implement end-to-end data governance, lineage tracking and security matrix enforcement utilizing Databricks Unity Catalog alongside Datasphere Spaces and scoping rules.
• Performance Optimization: Monitor and tune data transfer rates, cluster sizes, and query execution plans across both cloud platforms to minimize egress costs and maximize speed.

Must Have Skills:

Technical Core

• SAP Datasphere Expertise: 2 years of production experience with SAP Datasphere, including Space Management, Data Builder, Business Builder, and integration with S/4HANA or BW/4HANA.
• Databricks Mastery: 3+ years of hands-on experience building enterprise-grade lakehouses using Databricks, PySpark, and Delta Lake.
• Advanced Data Ingestion: Deep understanding of modern integration patterns between SAP cloud products and hyperscaler storage (ADLS Gen2, AWS S3) via Kafka, SAP FedML, or OData/CDI streaming.
• Data Governance: Strong experience with Unity Catalog for managing cross-platform data access and data lineage.
• Cross-Ecosystem Fluency: Ability to speak the language of both traditional SAP functional teams (ABAP, S/4 schemas) and modern cloud data science teams.
• Agile & DevOps: Experience with CI/CD tools (Git, Azure DevOps, GitHub Actions) adapted for Databricks notebooks and Datasphere transports.
• Communication: Exceptional communication skills to present architectural decisions to executive stakeholders and technical teams alike.