Lead I - Data Engineering

Job Summary Responsibilities

• Hands-on Migration: Execute the moving of data assets, schemas, tables, and notebooks from the source tenant to the target environment.
• Pipeline Updates: Update and test data pipelines, changing storage paths, API endpoints, and connection strings to match the new unified architecture.
• Workspace Setup: Use Terraform and platform APIs to spin up new, unified workspaces, clusters, and warehouses according to the new company standards.
• Permissions Mapping: Recreate and map user groups, access permissions, and roles from the old systems into the new centralized governance model.
• Monitoring Validation: Run data validation tests and monitor migration logs to ensure zero data loss or downtime during cutovers.

Technical Requirements

• Cloud Foundations: Solid hands-on experience using AWS (S3, IAM, EC2, Lambda) in a multi-account setup.
• Platform Experience: Practical administration experience with Databricks (clusters, notebooks, jobs) OR Snowflake (warehouses, roles, storage integration).
• Automation Tools: Comfortable using Terraform for provisioning infrastructure and Git/CI/CD for deployment pipelines.
• Core Coding: Proficient in Python and SQL for writing tests, scripting automation tasks, and verifying data structures.

Experience and Qualifications

• 2-4 years of experience working in Data Engineering, Platform Operations, or Cloud Operations.
• Strong troubleshooting skills and the ability to work collaboratively to solve migration roadblocks.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.