Data Engineer

We are looking for a Data Engineer who can design and build scalable data platforms supporting large-scale enterprise and AI use cases. You'll work across AWS, Python, SQL, and Spark/PySpark, building modern data architectures spanning data lakes, warehouses, and lakehouse platforms. This is a high-ownership role with exposure to architecture, distributed systems, AI/ML teams, and global clients.

Responsibilities:

• Design and build scalable data pipelines and ETL/ELT workflows.
• Develop high-performance data solutions using Python, SQL, and PySpark/Spark.
• Design and implement AWS-based data architectures.
• Build and optimize batch and real-time data pipelines.
• Work with large-scale data lakes, data warehouses, and lakehouse platforms.
• Build reliable data platforms supporting AI/ML and analytics use cases.
• Implement data quality, monitoring, and pipeline reliability practices.
• Contribute to technical architecture and engineering best practices.
• Mentor junior engineers and contribute to technical decision-making.
• Collaborate closely with global clients and cross-functional teams.

Requirements:

• 8-12 years of experience in data engineering/big data engineering.
• Strong, hands-on AWS experience is mandatory.
• Strong cloud engineering experience.
• Strong proficiency in Python and SQL.
• Hands-on experience with PySpark / Spark.
• Strong ETL/ELT and data pipeline development experience.
• Experience with AWS services such as S3 Glue, EMR, Redshift, Athena, Lambda, and Kinesis.
• Experience with Snowflake, Databricks, Redshift, or similar data platforms.
• Strong understanding of data modelling and distributed systems.
• Experience with CI/CD and DevOps practices.
• Strong communication skills and experience working with global stakeholders.
• Comfortable working the night shift.
• Good to have: Palantir Foundry, Kafka, Airflow, Docker/Kubernetes, Terraform, and MLOps/AI data pipelines.