Senior Data Platform Engineer

Who we are

DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, and certificate lifecycle management, to secure infrastructure, software, devices, messages, AI content and agents. Learn why more than 100,000 organizations, including 90% of the Fortune 500, choose DigiCert to stop today’s threats and prepare for a quantum-safe future at  www.digicert.com

Job summary

We are looking for a Senior Data Platform Engineer to design, build, and operate the foundational data and ML infrastructure that powers analytics, reporting, and machine learning across DigiCert. This role sits at the intersection of data engineering and ML platform work — you will own the systems, pipelines, and tooling that data scientists, analysts, and engineers rely on every day. You bring deep expertise in Databricks, a strong engineering mindset, and hands-on experience building and maintaining ML pipelines in production. You will partner closely with data science, analytics, product, and business teams to deliver a platform that is reliable, governed, and built to scale.

What you will do

Design, build, and maintain scalable data and ML pipelines using Python and SQL, processing large-scale datasets across batch and streaming workloads

Own and evolve core platform infrastructure on Databricks — including Delta Lake table architecture, Unity Catalog governance, Databricks Workflows orchestration, and compute optimization

Build and maintain end-to-end ML pipelines: feature engineering, model training pipelines, experiment tracking (MLflow), and model deployment/serving infrastructure

Collaborate with data scientists to operationalize models — bridging the gap between experimentation and production-grade ML systems

Define and enforce data platform standards: ingestion patterns, data modeling conventions, medallion architecture (Bronze/Silver/Gold), and pipeline reliability practices

Implement data quality, observability, and monitoring frameworks to ensure platform health and data trustworthiness

Optimize pipelines for performance, cost, and reliability at scale using Spark and PySpark

Evaluate, integrate, and govern new platform tooling and data sources within the Databricks ecosystem

Contribute to architectural decisions and help drive the long-term data platform roadmap

Participate in code reviews, technical design discussions, and engineering standards

Mentor junior engineers and elevate overall platform and data engineering practices

Document platform architecture, pipeline design, and operational runbooks

What you will have

6+ years of experience in data engineering, data platform, or ML engineering roles

Strong proficiency in Python and SQL, with a track record of building production-grade data pipelines using both

Hands-on Databricks expertise: Delta Lake, Unity Catalog, Databricks Workflows, PySpark, and the Databricks ecosystem broadly

Experience building and maintaining ML pipelines in production — feature engineering, training pipelines, experiment tracking, and model deployment

Familiarity with MLflow or comparable experiment tracking and model registry tools

Experience working on cloud data platforms (AWS, Azure, or GCP)

Strong understanding of data modeling, dimensional design, and analytics-friendly data architecture

Experience with batch and incremental/CDC pipeline patterns

Proficiency with Git, version control, and CI/CD practices for data and ML workflows

Strong engineering judgment — you think about reliability, maintainability, and cost, not just correctness

Clear communication and comfort working with both technical and non-technical stakeholders

Nice to have

Experience with streaming or near real-time pipelines (Kafka, Kinesis, Spark Structured Streaming)

Familiarity with feature store platforms (Databricks Feature Store, Feast, or Tecton)

Experience with LLM pipelines, RAG architectures, or AI/BI tooling (Genie, AI Functions)

Knowledge of data quality and observability tooling (Great Expectations, Monte Carlo, etc.)

Exposure to dbt or similar SQL-based transformation frameworks

Infrastructure-as-code experience (Terraform, Databricks Asset Bundles)

Experience working in Agile or Scrum environments

Prior experience mentoring engineers or shaping platform standards

Benefits

Generous time off policies

Top shelf benefits

Education, wellness and lifestyle support

To protect candidate information and maintain a secure hiring process, all applications must be submitted through our careers portal. Resumes or CVs sent directly via email will not be reviewed or considered.

#LI-SD1