AWS Lead Data Engineer - R01564133

Lead Data Engineer

Primary Skills

• Athena, Step Functions, Spark - Pyspark, ETL Fundamentals, SQL (Basic + Advanced), Glue, Python, Lambda, Data Warehousing, EBS /EFS, AWS EC2, Lake Formation, Aurora, S3, Modern Data Platform Fundamentals, PLSQL, Data Modelling Fundamentals, Cloud front Specialization
• AWS Data EngineerIng Basic: Senior Data Engineer Job requirements

Job Description - AWS Senior Data Engineer Company: Brillio Technologies

Role: Senior Data Engineer

Experience: 5-10 Years

Location: India (Hybrid/Remote)

Employment Type: Full-time

Role Overview We are looking for a

Senior Data Engineer with strong Databricks and Big Data expertise to design, build, and optimize scalable data pipelines and platforms. The ideal candidate will have hands-on experience with

Spark, PySpark, AWS ecosystem, and modern data architectures , along with exposure to APIs and real-time data processing.

Key Responsibilities

• Design and build scalable data pipelines using Spark / PySpark on Databricks
• Develop and optimize ETL/ELT workflows for large-scale data processing
• Work with AWS EMR, S3, and Hadoop ecosystem for distributed data processing
• Build and maintain data lake and data warehouse solutions
• Develop and integrate APIs for data ingestion and consumption
• Implement data processing workflows using Airflow / Autosys
• Optimize data performance using partitioning, caching, and query tuning
• Handle structured and semi-structured data from multiple sources
• Ensure data quality, governance, and reliability of pipelines
• Collaborate with cross-functional teams (Analytics, Product, Engineering)

Technical Skills (Must-Have)

• Strong expertise in: + Apache Spark / PySpark + Databricks (mandatory) + SQL (PostgreSQL or similar)
• Hands-on experience with: + AWS EMR, S3 + Hadoop, Hive ecosystem
• Programming skills: + Python (must-have) + Scala (good exposure)
• Strong knowledge of: + UNIX / Shell scripting + ETL pipelines and data engineering fundamentals

Additional Skills

• Experience with: + Elasticsearch (data storage & retrieval) + Workflow orchestration tools (Airflow, Autosys)
• Exposure to: + API development & integration + Version control tools (Git / SVN)
• Basic knowledge of HTML (for web/data tasks)

Experience Required

• Minimum 5+ years of Data Engineering experience
• Strong hands-on experience in Databricks + Big Data stack
• Proven experience working on large-scale distributed systems

Education

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field

Why Join Brillio

• Work on cloud-native, large-scale data platforms
• Exposure to cutting-edge Databricks & AWS ecosystems
• Collaborative and high-performance engineering culture