Technical Lead - Data Engineering (AWS)

Role Objective

We are seeking a highly skilled Lead Data Engineer / Tech Lead - Data Engineering to join our dynamic team. In this role, you will not only design and develop our cloud-based data infrastructure but also lead a team of Data Engineers to deliver robust and scalable data solutions for our BFSI customers. As a Tech Lead, you will work at the intersection of cloud technologies, data engineering, and the BFSI domain, while guiding the team to ensure project success.

Key Responsibilities:

• Lead, mentor, and manage a team of Data Engineers to deliver high-quality data solutions.
• Design, develop, and implement cloud-based data pipelines, ETL processes, and data integration solutions using platforms AWS.
• Collaborate with cross-functional teams to understand data requirements and design scalable data models and architectures that align with BFSI industry needs.
• Optimize data storage, processing, and retrieval for maximum performance and cost-efficiency in cloud environments.
• Implement data security and compliance measures to ensure the protection and integrity of sensitive BFSI data.
• Work closely with data scientists and analysts to enable seamless access to high-quality data for analytical purposes.
• Troubleshoot and resolve data-related issues, ensuring data availability and reliability for BFSI customers.
• Stay updated on industry best practices, emerging cloud technologies, and trends in the BFSI sector to drive continuous improvement and innovation.
• Manage project timelines, delegate tasks, and ensure the successful delivery of data engineering projects.

Qualifications & Experience:

• Minimum 6 years experience with good Hands on Pyspark, Airflow, AWS.
• Advanced SQL and PySpark expertise; strong understanding of Data Engineering concepts (DW, Data Lake, OLTP vs OLAP, etc.)
• API development experience (preferably in Python)
• Familiarity with Docker and Kubernetes
• Experience with Airflow
• Exposure to Hudi, Iceberg, or Delta Lake
• Strong AWS project experience (should also be adaptable to other cloud platforms if required)
• Proven experience in leading and managing a team of Data Engineers in previous roles.
• Bachelor's degree in Engineering, or a related field
• Proficiency in programming languages like Python, Java, or spark for data processing and ETL operations.
• Experience with big data technologies (e.g., Hadoop, Spark) and data processing frameworks.
• Familiarity with BFSI industry data standards, compliance requirements, and security protocols.
• Excellent problem-solving skills and the ability to work in a collaborative, cross-functional team environment