Big Data Developer

Big Data Developer Spark | Hadoop | Java/Python | SQL

Company: Spark Techwave InnovationZ Pvt Ltd
Job Type: Full-Time
Experience: 26 Years
Location: Chennai / Remote / Hybrid
Department: Data Engineering / Big Data

Job Summary

Spark Techwave InnovationZ Pvt Ltd is looking for an experienced Big Data Developer to design, develop, and maintain scalable applications and data-processing solutions using modern Big Data technologies.

The ideal candidate should have strong programming experience with Java and/or Python, along with hands-on experience in Apache Spark, Hadoop, SQL, distributed data processing, and Big Data application development.

Key Responsibilities

• Design and develop scalable Big Data applications and data-processing solutions.
• Develop data-processing applications using Apache Spark / PySpark.
• Write efficient and optimized code for processing large datasets.
• Develop data ingestion, transformation, and processing workflows.
• Work with Hadoop ecosystem technologies for large-scale data processing.
• Develop and optimize SQL queries for data extraction and transformation.
• Implement batch and real-time data-processing solutions.
• Troubleshoot application, data, and performance-related issues.
• Optimize Spark jobs and distributed processing applications.
• Integrate data from multiple sources and systems.
• Work with REST APIs and external data sources where required.
• Perform data validation and ensure data-processing accuracy.
• Participate in code reviews and follow software development best practices.
• Write unit tests and maintain technical documentation.
• Collaborate with Data Engineers, Data Scientists, Analysts, QA Engineers, and other development teams.
• Support deployment and maintenance of Big Data applications.

Required Skills

• Strong programming experience in Java and/or Python.
• Hands-on experience with Apache Spark / PySpark.
• Strong knowledge of SQL.
• Good understanding of Big Data concepts and distributed computing.
• Experience working with large datasets.
• Understanding of Hadoop ecosystem.
• Experience developing data-processing applications.
• Good knowledge of object-oriented programming concepts.
• Strong debugging and problem-solving skills.
• Familiarity with Linux/Unix environments.
• Experience with Git or other version-control systems.

Big Data Technologies

Experience with one or more:

• Apache Spark
• PySpark
• Hadoop
• HDFS
• Hive
• Kafka
• HBase
• Sqoop
• Spark Streaming
• Databricks

Cloud Technologies – Good to Have

Experience with one or more:

• AWS
• Microsoft Azure
• Google Cloud Platform
• AWS EMR
• AWS Glue
• Azure Databricks
• Azure Data Factory
• Google Dataproc

Database Skills

Experience with one or more:

• MySQL
• PostgreSQL
• SQL Server
• Oracle
• MongoDB
• Cassandra

Additional Skills – Good to Have

• REST APIs
• Docker
• CI/CD
• Kubernetes
• Jenkins
• Scala
• Airflow
• Data Warehousing
• Data Modeling
• Microservices

Education

Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related technical field.

Experience

2–6 years of relevant experience in Big Data Development, Big Data Engineering, Data Engineering, or distributed application development.

Preferred Candidate Profile

Candidates with hands-on experience in:

• Big Data application development
• Apache Spark / PySpark
• Java / Python
• Hadoop
• SQL
• Distributed data processing
• Batch and streaming data processing
• Big Data application optimization
• Cloud-based Big Data platforms

Naukri Key Skills

Big Data Developer, Big Data, Apache Spark, PySpark, Hadoop, Java, Python, SQL, Hive, HDFS, Kafka, Spark Streaming, Databricks, ETL, Data Processing, Distributed Computing, AWS, Azure, GCP

Interested candidates can apply with their updated resume at [email removed]