Big Data Engineer (5-15 Years)

Position: Big Data Developer
Experience: 5 to 15 Years
Location: PAN India
Employment Type: Full-Time

About the Role

We are looking for experienced Big Data Developers to design, build, and optimize large-scale data processing systems and analytics platforms. The ideal candidate will possess strong expertise in Big Data technologies, distributed computing, cloud platforms, and data engineering practices to support enterprise data-driven initiatives.

Key Responsibilities

• Design, develop, and maintain scalable Big Data solutions for processing and analyzing large volumes of structured and unstructured data.
• Build and optimize data pipelines, ETL/ELT processes, and real-time streaming applications.
• Develop data ingestion frameworks from multiple sources including APIs, databases, files, and streaming platforms.
• Implement batch and real-time data processing solutions using Big Data technologies.
• Collaborate with Data Architects, Data Scientists, Business Analysts, and stakeholders to understand business requirements.
• Ensure data quality, integrity, governance, and security across the data ecosystem.
• Optimize data processing performance and troubleshoot production issues.
• Participate in architecture reviews, code reviews, and technical discussions.
• Support cloud migration and modernization initiatives.
• Develop and maintain technical documentation and operational procedures.

Required Skills

Big Data Technologies

• Strong hands-on experience with:
• Hadoop Ecosystem
• Apache Spark (Core, SQL, Streaming)
• Hive
• HDFS
• YARN
• Experience developing distributed data processing applications.

Data Engineering

• Expertise in ETL/ELT development.
• Strong knowledge of data modeling and data warehousing concepts.
• Experience with large-scale data transformation and migration projects.
• Understanding of data governance and data quality frameworks.

Programming Skills

• Strong proficiency in:
• Python
• Scala
• Java
• SQL
• Experience with scripting and automation.

Streaming Technologies

• Experience with:
• Apache Kafka
• Spark Streaming
• Apache Flink (Preferred)
• Apache NiFi

Cloud Technologies

• Experience with one or more cloud platforms:
• AWS
• Azure
• Google Cloud Platform (GCP)

Database Technologies

• Strong experience with:
• SQL Server
• Oracle
• PostgreSQL
• MySQL
• MongoDB
• Cassandra (Preferred)

Tools & Technologies

• Apache Spark
• Hadoop
• Hive
• Kafka
• Scala
• Databricks
• Snowflake
• AWS
• Azure

Notice Period: Immediate to 30 Days Preferred

Keywords: Big Data, Spark, Hadoop, Kafka, Hive, Databricks, Data Engineering, Python, Scala, AWS, Azure, GCP, Snowflake, Airflow, ETL, Data Lake, Data Warehouse, Real-Time Streaming.

Provide your feedback on BizChat