Spark Scala Developer

Educational Requirements

• Bachelor of Engineering
• BTech
• BCA
• BSc
• MTech
• MCA
• MSc

Service Line Responsibilities

• Design and build scalable big data processing applications.
• Strong experience in Apache Spark, Scala, and distributed data processing; work on building high-performance data pipelines for analytics and data engineering use cases.
• Develop and maintain data pipelines using Apache Spark (Scala).
• Process large-scale datasets in distributed environments.
• Implement batch and real-time data processing solutions.
• Write efficient and scalable code using Scala.
• Work extensively with Spark Core, Spark SQL, and Spark Streaming.
• Optimize Spark jobs for performance and resource utilization.
• Ingest data from various sources: Databases (RDBMS, NoSQL), APIs, File systems (HDFS, S3).
• Build ETL/ELT pipelines and data transformation workflows.
• Tune Spark applications (partitioning, caching, memory tuning).
• Optimize queries using Spark SQL.
• Improve job execution time and reduce resource consumption.
• Work with data engineers, data scientists, and business analysts.
• Translate business requirements into scalable data solutions.
• Participate in code reviews and architecture discussions.
• Monitor data pipelines and Spark jobs.
• Debug job failures and performance bottlenecks.
• Ensure reliability and data quality in pipelines.

Technical and Professional Requirements

• Primary skills: Big Data - Data Processing - Spark; Functional Programming - Scala.

Preferred Skills

• Functional Programming - Scala
• Big Data - Data Processing - Spark