Senior Data Engineer

Job Title: Senior Data Engineer Lakehouse Platform

Experience: 8+ years
Employment Type: Contractual
Work Mode: 100% Remote
Working Hours: 6:00 PM – 3:00 AM IST (US Shift)

About the Role

We are looking for an experienced Senior Data Engineer to design, build, and scale a production-grade data lakehouse platform on AWS. The role involves building a reliable, governed, and scalable data foundation to support enterprise analytics, financial reporting, and AI-driven applications.

The ideal candidate will have strong hands-on expertise in AWS data services, Apache Iceberg, dbt, SQL Server CDC pipelines, and modern data architecture, with a focus on data accuracy, reliability, performance, and cost optimization.

Key Responsibilities

• Design and scale a Medallion Lakehouse Architecture (Bronze, Silver, and Gold) using Apache Iceberg, Amazon S3, Athena, Redshift Serverless, dbt Cloud, and AWS Lake Formation.
• Build and maintain AWS DMS-based Change Data Capture (CDC) pipelines from SQL Server databases.
• Develop reliable SQL transformations for complex business processes, including billing, invoicing, time tracking, and financial reporting.
• Implement SCD Type 2, snapshots, incremental processing, and historical data tracking.
• Design scalable multi-tenant data architectures with appropriate tenant isolation, partitioning, and skew management.
• Manage dbt Cloud workflows, including semantic layers, dbt Mesh, MetricFlow, testing, and incremental models.
• Build governed data models and metrics to support analytics and AI-agent consumption, including MCP integrations where applicable.
• Establish data quality checks, reconciliation processes, freshness SLAs, observability, and data lineage.
• Optimize query performance, storage costs, Iceberg compaction, and data processing efficiency.
• Manage Iceberg operations, including snapshot expiry, time travel, and partition evolution.
• Collaborate with application, analytics, and AI teams to translate business requirements into reliable data products.
• Troubleshoot production issues and ensure data accuracy, availability, and consistency.

Mandatory Skills and Experience

• 10+ years of experience building and managing production-grade data platforms.
• At least 3 years of experience with lakehouse architectures using Apache Iceberg, Delta Lake, or Apache Hudi.
• Strong hands-on experience with AWS DMS and CDC pipelines from SQL Server databases.
• Advanced dbt Cloud skills, including semantic layer, dbt Mesh, MetricFlow, snapshots, SCD2, testing, and incremental models.
• Production experience with Amazon S3, Athena, Redshift Serverless, and AWS Lake Formation.
• Strong SQL Server and T-SQL skills, including the ability to understand complex legacy application logic.
• Expertise in dimensional modeling, data warehousing, and Kimball/Inmon modeling approaches.
• Experience with multi-tenant SaaS data platforms at scale.
• Hands-on knowledge of Iceberg compaction, snapshot management, partition evolution, and time travel.
• Proven experience with financial data reconciliation and business-critical data accuracy.
• Strong analytical, troubleshooting, and performance-tuning skills.

Good to Have

• Experience exposing governed metrics or dbt semantic layers to AI agents through MCP.
• Familiarity with AWS Glue, Step Functions, EventBridge, MWAA, and EMR Serverless.
• Experience with zero-ETL integrations, such as Aurora to Redshift or OpenSearch.
• Experience in fintech, legal technology, or other regulated industries.
• Knowledge of query-plan analysis, Redshift sort keys, and Iceberg file-size optimization.
• Exposure to LLM-powered applications and AI-enabled data platforms.

Employment Terms

• Contractual engagement with a fully remote working arrangement.
• Candidates must be available to work from 6:00 PM to 3:00 AM IST.
• Candidates must be willing to commit to a minimum engagement of one year.
• Immediate joiners or candidates with a short notice period will be preferred.
• Candidates must be comfortable working on a contractual basis and independently owning production data infrastructure.

How to Apply

Interested candidates are requested to share their updated CV along with the following details:

1. Current CTC
2. Expected CTC
3. Notice period
4. Willingness to commit for a minimum of one year
5. Willingness to work on a contractual basis
6. Willingness to work fully remotely
7. Availability for the US shift (6:00 PM – 3:00 AM IST)
8. Earliest possible joining date

Preference will be given to immediate joiners and candidates who meet the technical requirements and contractual commitment.