Senior Analytics Engineer

Position Summary

We are looking for an experienced Senior Analytics Engineer with deep expertise in Databricks, data modeling, and AWS to design, build, and scale the analytics layer that sits between raw curated data and business-facing insights for commercial analytics and reporting. In this role, you will own the transformation logic, dimensional/semantic models, and metrics definitions that turn refined-layer datasets into trusted, self-serve analytics products on the Databricks Lakehouse. You will set standards for data modeling and testing, mentor and work hands-on alongside junior developers to help them grow, and partner directly with analysts, data scientists, and business stakeholders to define metrics, data models, and governed datasets that power dashboards, self-service BI, and advanced analytics. You will also drive technical decisions on Databricks and AWS architecture, performance, scalability, and cost optimization across the analytics stack, while confidently communicating technical concepts and trade-offs to business audiences.

Key Responsibilities

• Design, build, and own scalable analytics-layer transformations (marts, semantic models, and metrics) on Databricks, using strong dimensional and analytical data modeling practices and modern transformation frameworks (e. g. , dbt on Databricks).
• Architect analytics-ready data models (star/snowflake schemas, conformed dimensions, fact tables) that translate complex commercial business logic into consistent, reusable data structures.
• Own performance tuning and cost optimization of Databricks workloads, including cluster configuration, Delta Lake optimization (Z-ordering, partitioning, OPTIMIZE/VACUUM), Unity Catalog, and job/workflow orchestration.
• Design and manage AWS-based data architecture underpinning the Lakehouse (e. g. , S3 storage layout, IAM roles/policies, Glue, Lambda, and networking/security fundamentals) in partnership with cloud/platform teams.
• Architect and optimize datasets for large-scale, structured and semi-structured commercial datasets (e. g. , sales, claims, patient, or similar), including complex cross-domain joins and slowly changing dimensions (SCD).
• Establish and enforce testing, validation, documentation, and CI/CD practices for Databricks-based analytics code, ensuring data quality, lineage, and reliability at scale.
• Lead technical design reviews and set best practices for data modeling, Databricks architecture, and AWS resource usage; mentor and provide technical guidance to data and analytics engineers.
• Mentor and coach junior developers day-to-day - pairing on code, reviewing pull requests, and working alongside them on shared deliverables to build their skills in data modeling, Databricks, and AWS.
• Act as a primary point of contact for business stakeholders, clearly explaining technical trade-offs, data model decisions, and dataset limitations in business-friendly terms.
• Partner closely with analysts, data scientists, and business stakeholders to translate ambiguous business questions into well-modeled, analytics-ready data products, and define dataset readiness and adoption criteria.
• Apply and champion data governance practices on Databricks and AWS, including documentation, lineage, access controls (Unity Catalog/IAM), and compliant handling of sensitive/regulated data.

Skills & Competencies

• Expert, hands-on experience with Databricks (notebooks, jobs/workflows, clusters, Unity Catalog) and Delta Lake (ACID tables, incremental processing, upserts/merge, Z-ordering, partitioning, performance tuning) - this is a core requirement.
• Strong expertise in data modeling for analytics - dimensional/Kimball-style star and snowflake schemas, conformed dimensions, fact tables, SCD handling, and lakehouse concepts (medallion architecture; bronze/silver/gold layers) - this is a core requirement.
• Solid, hands-on experience with AWS as the underlying cloud platform (S3, IAM, Glue, Lambda, networking/security fundamentals) supporting a Databricks Lakehouse - this is a core requirement.
• Advanced proficiency in SQL and Python for data transformation, modeling, and validation at scale.
• Hands-on experience with analytics engineering frameworks (e. g. , dbt) and building governed semantic/metrics layers on top of Databricks.
• Experience with workflow orchestration (e. g. , Databricks Workflows, Airflow) and engineering best practices (Git/version control, code review, CI/CD).
• Strong grounding in data governance and secure data handling (documentation, lineage, access controls via Unity Catalog/IAM, PII/PHI awareness).
• Proficiency with BI/visualization tools (Tableau/Power BI) and enabling self-service analytics for business users.
• Demonstrated ability to mentor and coach junior developers, including hands-on pairing and code review, while working alongside them as part of the same delivery team.
• Excellent verbal and written communication skills, with the ability to explain technical concepts clearly and confidently to non-technical business stakeholders.
• Strong stakeholder-management skills; comfortable leading conversations with business partners to gather requirements, manage expectations, and present findings.

Qualifications & Experience

• Bachelors or Masters degree in Computer Science, Engineering, Information Systems, Statistics/Mathematics, or a related field (or equivalent practical experience).
• 6+ years of hands-on experience in analytics engineering, data engineering, or related roles, with demonstrated depth in Databricks, data modeling, and AWS.
• Required: significant production experience building and optimizing pipelines and data models on Databricks and Delta Lake at scale.
• Required: strong, demonstrable data modeling experience (dimensional modeling, schema design) for analytics-ready datasets.
• Required: working experience with AWS services (S3, IAM, Glue, Lambda, or similar) in a production data platform.
• Preferred: experience with dbt on Databricks and Unity Catalog for governance and access control.
• Required: demonstrated experience mentoring junior developers and working alongside them in a hands-on capacity (pairing, reviews, shared delivery).
• Preferred: experience presenting to or working directly with business stakeholders (e. g. , requirements gathering, readouts, roadmap discussions).
• Preferred: prior experience leading engineers.
• Added advantage: understanding of the pharma/biopharma domain and commercial datasets (e. g. , claims, sales, payer, patient, HUB/specialty pharmacy), including common identifiers and integration challenges