Senior Data Engineer

Short Description
Description
We are seeking a highly accomplished and visionary Advanced Data Engineer / Technical Anchor to join our team. In this role, you will serve as the technical lead and architectural authority, designing enterprise-grade Cloud Data Solution Architectures leveraging Python, Cloud Dataproc (Spark), BigQuery, Astro (Airflow), and Terraform. You will act as a strategic partner between engineering teams and business leadership, establishing organizational CI/CD standards (Tekton, GitHub) and security scanning controls (Checkmarx, FOSSA, SonarQube). You will govern Cloud Run audit signals and GCS log archival frameworks, and mentor engineering talent to maximize the long-term strategic value of our data platform.
Responsibilities

• Technical Anchor & Solution Architecture: Act as the lead engineer and domain expert, designing enterprise-grade Cloud Data Solution Architectures, implementing advanced design patterns, and guiding technology selection across the platform.
• Strategic Roadmap Alignment: Partner with business and engineering leaders to align technical roadmaps with organizational goals. Provide strategic advice on "build vs. buy" decisions to maximize long-term business value.
• IaC & CI/CD Strategy: Establish organizational standards for Infrastructure as Code using Terraform. Standardize automated release pipelines via GitHub and Tekton, embedding mandatory quality, security, and license compliance controls (Checkmarx, FOSSA, SonarQube).
• High-Performance Distributed Processing: Architect and optimize large-scale real-time and batch data processing systems leveraging BigQuery, Dataflow, Dataproc, Cloud Run, Pub/Sub, and distributed computing frameworks on GCP.
• Platform Health & Quality Metrics: Establish organizational coding patterns, reusable frameworks, and SLO compliance. Introduce and drive Software Engineering (SWE) metrics, including DORA metrics (lead time, deployment frequency, MTTR, change-failure rate), code coverage, cyclomatic complexity targets, and P99 latency standards.
• Data Governance & Enterprise Security: Drive the strategy and implementation for enterprise data governance, metadata management, data lineage, and strict GCP security controls (IAM, VPC Service Controls, encryption).
• Resilience, Observability & Incident Leadership: Establish deployment and observability standards; lead critical incident resolution and Root Cause Analysis (RCA) to implement permanent solutions and address technical debt.
• Mentorship & Culture: Mentor engineers across teams, conduct high-impact code reviews (Code Guardians), drive cultural change, and represent organizational technical excellence internally and externally (tech talks, blogs, open-source).

Qualifications

• Education & Experience: Bachelors or Masters degree in Computer Science, Software Engineering, or a related field, with extensive experience in enterprise data platform architecture, distributed systems, and technical leadership/anchor roles.
• Data Architecture & Processing: Expert mastery of Python, distributed computing with Spark on Cloud Dataproc, and advanced BigQuery data modeling, query optimization, and cost management.
• Programming & Systems Design: Expert mastery of Python, Java, or Scala for distributed computing, real-time streaming, and complex data pipeline creation.
• CI/CD & Security Frameworks: Comprehensive experience integrating Tekton, GitHub, SonarQube, Checkmarx, and FOSSA into automated release gates. Deep knowledge of enterprise GCP security (IAM, Secret Manager) and audit architectures (Cloud Run, Cloud Storage).
• Data Governance & Security Frameworks: Deep experience building data governance, metadata management, data lineage, and security infrastructure within GCP (VPC SC, IAM, KMS).
• Engineering Metrics & Quality Control: Proven experience embedding SWE-centric operational metrics (DORA metrics, code coverage, cyclomatic complexity, latency optimization) into platform operations.
• Executive Leadership & Influence: Exceptional communication skills with a proven track record of influencing cross-team strategies, mentoring engineering talent, driving platform health initiatives, and resolving complex technical blockers.