Data Engineer

Role & responsibilities

8-10 years of overall IT experience, with strong experience in data engineering, analytics engineering, cloud data platforms, or observability data platforms.

•           Hands-on experience with enterprise-scale data modeling and pipeline implementation.

•           Strong understanding of data architecture, data quality, pipeline reliability, and serving-layer design.

**Role Summary**

We are seeking an experienced **Observability Data Engineer** to design, build, and optimize data pipelines and data models that power enterprise observability, operational analytics, and AIOps initiatives. The ideal candidate will have strong expertise in **dbt**, **Snowflake**, **AWS S3**, **Python**, and modern data engineering practices, with the ability to transform telemetry and operational data into scalable, reliable, and actionable datasets.

This role will focus on developing observability data products, building reliable data pipelines, implementing data contracts, and enabling analytics and visualization solutions that support platform engineering and operational excellence.

**Key Responsibilities**

**Data Modeling & Transformation**

•       Design, develop, and maintain scalable data models using **dbt**.

•       Build curated observability data layers supporting analytics, reporting, alerting, and AI-driven use cases.

•       Create reusable transformation frameworks and semantic models for operational data.

•       Optimize data models for performance, maintainability, and scalability.

**Data Platform Engineering**

•       Develop and maintain data pipelines between source systems, telemetry platforms, and analytics environments.

•       Build and manage observability data serving layers utilizing:

o   **Snowflake**

o   **Amazon S3**

•       Design data ingestion, transformation, and consumption architectures.

•       Ensure high data quality, availability, and reliability across the platform.

•       Understand Data Integrations between AWS, Snowflake & multiple different data sources, structured & unstructured data  /Design Data Model & Exp in analytics tech stack like Power BI or Snowflake.

**Pipeline Architecture & Data Contracts**

•       Define and implement **data contracts** between producing and consuming systems.

•       Design robust and scalable data pipelines using modern data engineering best practices.

•       Establish standards for schema evolution, metadata management, and data governance.

•       Collaborate with platform and observability teams to ensure reliable data flows.

**Observability Analytics Enablement**

•       Support creation of operational and observability datasets from logs, metrics, traces, events, and infrastructure telemetry.

•       Develop datasets that enable incident analysis, trend analysis, capacity planning, and operational reporting.

•       Partner with engineering and observability teams to understand analytical requirements.

**Python Development & Automation**

•       Develop data processing utilities, validation frameworks, and automation solutions using **Python**.

•       Automate data quality checks, reconciliation processes, and monitoring workflows.

•       Build frameworks for pipeline testing, validation, and operational support.

**Visualization & Data Consumption**

•       Support integration of curated datasets with **Grafana Scenes** and other observability visualization platforms.

•       Develop and optimize data sources for dashboards, operational analytics, and executive reporting.

•       Enable self-service data access and analytics for engineering and operations teams.

**Data Governance & Reliability**

•       Implement data quality, lineage, and monitoring frameworks.

•       Ensure compliance with enterprise data management standards.

•       Develop documentation for data models, pipelines, and operational processes.

•       Support troubleshooting and root cause analysis of data platform issues.

****

**Required skills:**

•       Observability

•       Grafana Cloud

•       IT observability

•       AWS OpenTelemetry

•       Snowflake

•       AWS

•       Amazon S3

Python