Senior Data Engineer - Full Stack

Job summary

We are looking for a Sr. Data Engineer - Full Stack to design and deliver end-to-end data solutions on Databricks - from stakeholder discovery and data ingestion through modeling, APIs, applications, and production operations.This hands-on role combines deep data engineering expertise with strong stakeholder partnership, product thinking, and end-to-end solution ownership.You will work directly with business and technical teams to clarify ambiguous problems, rapidly prototype solutions, and turn validated concepts into secure, scalable, and reliable data products.You will bridge data engineering, analytics, software engineering, and business teams to make trusted data actionable.

What you will do

• Partner directly with business and technical stakeholders to understand workflows, define desired outcomes, and translate ambiguous needs into technical requirements and delivery plans.
• Design, build, and maintain end-to-end data products on Databricks, spanning ingestion, Delta Lake storage, transformation, serving, APIs, and user-facing experiences.
• Develop and operate reliable batch, incremental, and streaming pipelines using SQL, Python, PySpark, Kafka, and Databricks-native capabilities.
• Design event-driven and near-real-time data solutions that integrate with operational systems and downstream consumers.
• Build backend services, APIs, and integrations that make governed data available to applications and operational workflows.
• Develop lightweight applications, dashboards, and interfaces in partnership with product, analytics, BI, and user-experience teams.
• Rapidly prototype solutions, validate them through stakeholder feedback, and prepare successful concepts for production use.
• Create scalable data models and curated datasets that support analytics, reporting, AI/ML, and operational decision-making.
• Implement data-quality, security, lineage, and governance controls using Databricks and Unity Catalog.
• Establish automated testing, CI/CD, monitoring, alerting, and documentation across the data-product lifecycle.
• Optimize pipelines, streaming workloads, queries, services, and applications for reliability, performance, scalability, and cost.
• Diagnose and resolve issues across source systems, streaming platforms, pipelines, data models, APIs, applications, and downstream consumers.
• Collaborate with platform and product engineering teams to turn recurring stakeholder needs into reusable capabilities.
• Lead technical design and code reviews, mentor engineers, and help elevate full-stack data-engineering practices.

What you will have

• 5+ years of experience in data engineering, software engineering, or a related role, including ownership of production data solutions.
• Strong proficiency in SQL, Python, and PySpark, with experience building reliable, production-grade pipelines and data products.
• Hands-on experience with Databricks, Apache Spark, Delta Lake, and Unity Catalog or comparable data-governance capabilities.
• Experience designing and supporting streaming or near-real-time data pipelines using Kafka, Kinesis, Event Hubs, or similar event-streaming technologies.
• Strong understanding of event-driven architecture, message processing, schema evolution, data consistency, and streaming reliability.
• Experience delivering full-stack solutions that include data pipelines, backend services or APIs, and lightweight user-facing applications.
• Experience building REST APIs, services, and integrations using Python frameworks such as FastAPI, Flask, or comparable technologies.
• Experience with AWS, Azure, or GCP and cloud-native architecture patterns.
• Strong understanding of data modeling, data warehousing, distributed processing, and analytics-friendly data design.
• Experience with Git, automated testing, CI/CD, monitoring, and production-deployment practices.
• Demonstrated ability to work directly with stakeholders, navigate ambiguity, and translate business problems into practical technical solutions.
• Strong communication, technical leadership, problem-solving, and end-to-end ownership skills.
• Ability to balance rapid delivery with maintainability, security, governance, and operational reliability.

Nice to have

• Experience building applications with React or another modern frontend framework.
• Familiarity with infrastructure as code, containerization, and automated cloud deployment.
• Experience supporting AI/ML pipelines, feature engineering, retrieval systems, or generative AI use cases.
• Experience with data observability, platform engineering, or data-product management practices.
• Working knowledge of JavaScript or TypeScript and a modern frontend framework such as React, or comparable data-application experience.
• Background in forward-deployed engineering, solutions engineering, technical consulting, or another stakeholder-embedded delivery role.
• Experience designing reusable data platforms, frameworks, or self-service capabilities.
• Prior experience mentoring engineers and contributing to engineering standards.
• Experience working in Agile or Scrum environments.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.