Data Engineer

Responsibilities:

• Design, develop, and maintain production-grade ETL/ELT pipelines for large-scale data processing.
• Build reliable and scalable data workflows using orchestration tools such as Airflow, Dagster, or Prefect.
• Develop efficient data transformation and analytics workflows using Pandas, Polars, DBT, and DuckDB.
• Work with event-driven and event-based ETL architectures for real-time and asynchronous data processing.
• Design and optimise data models across databases such as Snowflake, PostgreSQL, MSSQL, and similar technologies.
• Build resilient data pipelines using Databricks, Delta Lake, Auto Loader, and workflow orchestration.
• Develop cloud-native data solutions across AWS, Azure, or GCP.
• Implement data quality checks, monitoring, observability, and pipeline reliability mechanisms.
• Leverage AI tools such as Claude, GitHub Copilot, and similar tools to accelerate software development and improve engineering productivity.
• Explore and implement LLM-based solutions, AI agents, and AI-powered automation for data engineering workflows.
• Work with technologies such as LangGraph, Claude Code, LLM APIs, and prompt engineering where applicable.
• Collaborate with product, engineering, data science, and other cross-functional teams in a distributed work environment.
• Troubleshoot complex data and engineering problems and develop scalable, maintainable solutions.
• Contribute to technical strategy, architecture decisions, and product roadmap discussions.

Requirements:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• 3+ years of software engineering experience, with a strong focus on data engineering.
• Strong programming skills in Python and SQL.
• Hands-on experience with Pandas and/or Polars.
• Experience building production-grade ETL/ELT pipelines.
• Experience with workflow orchestration tools such as Airflow, Dagster, or Prefect.
• Hands-on experience with DBT and DuckDB.
• Understanding of event-based/event-driven ETL architectures.
• Strong knowledge of database technologies and the ability to select the right database for different use cases.
• Experience with one or more of Snowflake, PostgreSQL, MSSQL, or similar databases.
• Strong problem-solving skills with the ability to work effectively in ambiguous environments.
• Excellent communication and collaboration skills.
• Genuine curiosity and passion for AI and its applications in data engineering.
• Experience using AI-assisted development tools such as Claude, GitHub Copilot, or similar platforms.