Lead II - Data Engineering

Job Summary

• Sr Data Platform Engineer Tenant Consolidation: Contribute to the architectural design and execution strategy to merge separate Databricks workspaces or Snowflake accounts into a unified platform.
• Metadata & Catalog Unification: Consolidate data catalogs, aligning separate Unity Catalogs or Snowflake Shares into a single, company-wide governance model.
• Security & Access Alignment: Audit and merge distinct security models, unifying IAM roles, active directory syncs, and row/column-level access controls.
• Pipeline & Code Migration: Refactor and migrate legacy ETL/ELT pipelines, Spark workflows, and orchestration routines to point to the new unified tenant.
• Cost & Asset Optimization: Identify overlapping data assets, compute resources, and storage tiers during the merge to optimize cloud spend.

Technical Requirements

• Cloud Infrastructure: Proven experience managing AWS ecosystems (S3, IAM, CloudTrail, VPC, Route53) with an emphasis on cross-account access.
• Data Platform Mastery: Deep expertise in Databricks (workspace administration, Delta Lake, Photon) OR Snowflake (multi-account management, Snowpark, Virtual Warehouses).
• Infrastructure as Code (IaC): Strong experience writing Terraform scripts to provision and replicate workspace configurations.
• Development Skills: Python and SQL skills, with experience auditing and refactoring code written by other teams.

Experience and Qualifications

• 4-6 years of experience in Data Platform Engineering, or Data Architecture.
• good to have Prior experience working on data platform migrations, cloud account consolidations,
• Ability to act as a technical lead, mapping out migration schedules and coordinating tasks with the mid-level engineer.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.