Big Data Engineer

We are seeking a highly skilled Java Spark Kafka Developer to design develop and support scalable highperformance data processing and streaming applications The ideal candidate should possess strong expertise in Java Apache Spark and Apache Kafka along with experience in distributed systems microservices architecture and cloudbased data platforms

Key Responsibilities

Design develop and maintain enterprisegrade applications using Java and related frameworks

Build realtime data streaming solutions using Apache Kafka

Develop batch and stream processing applications using Apache Spark

Design and implement scalable microservices and REST APIs

Develop Kafka producers consumers topics partitions and message processing pipelines

Optimize Spark jobs for performance scalability and reliability

Perform code reviews unit testing and support deployment activities

Collaborate with business stakeholders architects and crossfunctional teams to gather requirements and provide solutions

Troubleshoot production issues and implement performance improvements

Follow AgileScrum methodologies and DevOps best practices

Required Technical Skills

Core Java

Collections Framework

Multithreading Concurrency

Exception Handling

Design Patterns

JVM Performance Tuning

Apache Spark

Spark Core

Spark SQL

Spark Streaming Structured Streaming

DataFrame Dataset APIs

Performance Tuning

Spark Optimization Techniques

Apache Kafka

Kafka Architecture

Producers Consumers

Topic Management

Partitioning Replication

Consumer Groups

Kafka Streams

Schema Registry

Frameworks Technologies

Spring Boot

Spring MVC

Spring Data JPA

RESTful Web Services

Microservices Architecture

Hibernate JPA

Database Skills

Oracle SQL Server PostgreSQL

SQL Query Optimization

Stored Procedures

Database Design

Preferred Skills

Hadoop Ecosystem HDFS Hive

Kubernetes Docker

OpenShift

CICD tools Jenkins GitHub Actions

Cloud Platforms AWS Azure GCP

Elasticsearch

Unix Linux Shell Scripting

Educational Qualification

Bachelors Masters degree in Computer Science Information Technology Engineering or a related field

Soft Skills

Strong analytical and problem solving skills

Excellent communication and stakeholder management

Ability to work in a fast paced Agile environment

Strong collaboration and teamwork skills

Nice to have Certifications

Apache Kafka Certification

Databricks Spark Certification

Oracle Java Certification

AWSAzure Data Engineering Certification

Relevant Roles Java Developer Big Data Developer Data Engineer Streaming Data Engineer Backend Developer Kafka Developer Spark DeveloperProvide your feedback on BizChat

Skills

Mandatory Skills : Apache Spark, Java, Kafka