Senior Data Engineer - R01564133

Lead Data Engineer

Primary Skills

• Athena, Step Functions, Spark - Pyspark, ETL Fundamentals, SQL (Basic + Advanced), Glue, Python, Lambda, Data Warehousing, EBS /EFS, AWS EC2, Lake Formation, Aurora, S3, Modern Data Platform Fundamentals, PLSQL, Data Modelling Fundamentals, Cloud front
  
  

Specialization

• AWS Data EngineerIng Basic: Senior Data Engineer
  
  

Job requirements

Job Description - AWS Senior Data Engineer

Company: Brillio Technologies

Role: Senior Data Engineer

Experience: 5-10 Years

Location: India (Hybrid/Remote)

Employment Type: Full-time

Role Overview

We are looking for a Senior Data Engineer with strong Databricks and Big Data expertise to design, build, and optimize scalable data pipelines and platforms. The ideal candidate will have hands-on experience with Spark, PySpark, AWS ecosystem, and modern data architectures, along with exposure to APIs and real-time data processing.

Key Responsibilities

• Design and build scalable data pipelines using Spark / PySpark on Databricks
• Develop and optimize ETL/ELT workflows for large-scale data processing
• Work with AWS EMR, S3, and Hadoop ecosystem for distributed data processing
• Build and maintain data lake and data warehouse solutions
• Develop and integrate APIs for data ingestion and consumption
• Implement data processing workflows using Airflow / Autosys
• Optimize data performance using partitioning, caching, and query tuning
• Handle structured and semi-structured data from multiple sources
• Ensure data quality, governance, and reliability of pipelines
• Collaborate with cross-functional teams (Analytics, Product, Engineering)
  
  

Technical Skills (Must-Have)

• Strong expertise in:
• Apache Spark / PySpark
• Databricks (mandatory)
• SQL (PostgreSQL or similar)
• Hands-on experience with:
• AWS EMR, S3
• Hadoop, Hive ecosystem
• Programming skills:
• Python (must-have)
• Scala (good exposure)
• Strong knowledge of:
• UNIX / Shell scripting
• ETL pipelines and data engineering fundamentals
  

Additional Skills

• Experience with:
• Elasticsearch (data storage & retrieval)
• Workflow orchestration tools (Airflow, Autosys)
• Exposure to:
• API development & integration
• Version control tools (Git / SVN)
• Basic knowledge of HTML (for web/data tasks)
  
  

Experience Required

• Minimum 5+ years of Data Engineering experience
• Strong hands-on experience in Databricks + Big Data stack
• Proven experience working on large-scale distributed systems
  
  

Education

• Bachelor's or Master's degree in Computer Science, Engineering, or related field
  
  

Why Join Brillio

• Work on cloud-native, large-scale data platforms
• Exposure to cutting-edge Databricks & AWS ecosystems
• Collaborative and high-performance engineering culture