Data Engineer

Role & responsibilities

Data Pipeline Development

• Design and implement robust, scalable data pipelines using our modern data stack
• Build real-time streaming pipelines with Kafka and Change Data Capture (CDC)
• Develop batch processing workflows for data transformation and analytics
• Implement data quality frameworks and monitoring systems

Infrastructure & Architecture

• Architect and maintain our data lake infrastructure using Apache Iceberg
• Optimize data storage and retrieval across multiple data stores (PostgreSQL, Snowflake, ClickHouse)
• Design partitioning and sharding strategies for high-volume datasets.
• Data Integration & CDC

• Set up and maintain Change Data Capture (CDC) pipelines using Debezium

• Build real-time data synchronization between operational and analytical systems
• Handle schema evolution and ensure backward compatibility
• Implement idempotent data processing patterns
• Performance Optimization
• Optimize query performance across different data engines
• Implement efficient data compression and storage strategies
• Design and maintain data indexing strategies

• Monitor and troubleshoot performance bottlenecks
• Collaboration & Leadership

• Collaborate with data scientists, analysts, and product teams

• Mentor junior engineers and contribute to technical decision-making

• Participate in code reviews and maintain high engineering standards

• Document architectures, processes, and best practices

Preferred candidate profile

Advanced Technical Experience

Experience with multiple cloud providers (AWS, GCP, Azure)

Hands-on experience with Apache Iceberg in production environments

Experience with data mesh or federated data architectures

Knowledge of machine learning pipelines and feature stores

Experience with data quality tools (Great Expectations, Deequ, etc.)

Leadership & Collaboration

Experience mentoring junior engineers or leading technical initiatives

Strong communication skills with both technical and non-technical stakeholders

Experience working in agile development environments

Track record of improving data engineering practices and standards

Industry Experience

Experience in relevant industry: fintech, e-commerce, SaaS, etc.

Understanding of regulatory requirements (GDPR, SOX, PCI, etc.)

Experience with A/B testing and experimentation platforms