Data Engineer

Data Engineering Specialist

About the Role

We are looking for a Data Engineering Specialist to build and support reliable data solutions for BI, analytics, and AI use cases. The ideal candidate has strong Python and SQL skills, hands-on experience with data pipelines and cloud platforms, and experience working with BI and analytics platforms.

Key Responsibilities

• Design, build, and maintain scalable batch and streaming data pipelines across cloud-based data platforms.
• Develop data solutions using Python, SQL, and distributed processing technologies such as PySpark.
• Integrate data from APIs, databases, files, and event-based sources.
• Build reliable, analytics-ready datasets and data models for BI and analytical workloads.
• Develop and maintain workflows using Airflow or similar orchestration platforms.
• Work closely with BI and analytics teams to translate business requirements into trusted datasets and reporting solutions.
• Support BI platforms such as Power BI, Tableau, or QuickSight, including data models, reporting datasets, metrics, and data validation.
• Develop efficient SQL transformations and optimize queries, pipelines, and data models for performance, scalability, and cost.
• Implement data quality, validation, monitoring, alerting, and error-handling processes.
• Investigate production issues and data discrepancies, perform root-cause analysis, and drive resolution.
• Follow engineering, security, governance, and privacy best practices while maintaining appropriate technical documentation and operational runbooks.

Required Skills

• Strong Python and SQL programming skills with experience developing production-grade data solutions.
• Proven experience building and supporting ETL/ELT pipelines in production environments.
• Hands-on experience with cloud data engineering on AWS, GCP, Azure, or similar platforms.
• Experience working with cloud storage, data processing technologies, and cloud data warehouses.
• Good understanding of data warehousing and dimensional modeling, including facts, dimensions, and slowly changing dimensions.
• Experience with a workflow orchestration platform such as Airflow, Azure Data Factory, AWS Step Functions, or similar.
• Experience working with BI and analytics platforms, with an understanding of how data engineering solutions support reporting and business analytics.
• Experience with data transformation and testing frameworks such as dbt or similar tools.
• Working knowledge of Git, CI/CD, and Docker.
• Understanding of data quality, monitoring, governance, security, and production support.
• Strong problem-solving, troubleshooting, communication, and collaboration skills.

Good to Have

• Hands-on experience with Apache Airflow and Power BI.
• Experience with Kafka, Kinesis, CDC, Debezium, or Spark Structured Streaming.
• Experience with Databricks or other modern data platforms.
• Knowledge of Terraform and cloud security.
• Experience developing lightweight APIs using FastAPI, Flask, Django, or similar frameworks.
• Experience supporting data platforms for AI/ML or LLM applications.
• Experience working with regulated or sensitive data.
• Relevant cloud or data engineering certifications.
• Experience using AI/LLM tools to improve engineering productivity.