Scala Developer

Role Overview

Experience- 6-11 Years

Location-Mumbai

Work Mode- Hybrid

We are looking for a hands-on **Scala Developer** with strong experience in **Apache Spark, PySpark, HDFS and SQL** to design, develop and optimize scalable data processing solutions.

The ideal candidate should have strong Scala programming fundamentals, practical experience with Spark-based development, and the ability to independently understand business problems and translate them into scalable technical solutionsKey Responsibilities

- Understand business requirements and implement tactical or strategic technology solutions.

- Work closely with business and technology stakeholders to understand requirements and translate them into technical solutions.

- Define appropriate strategic solutions while identifying tactical approaches where required.

- Prepare technical design documents and solution documentation in an Agile/Scrum environment.

- Perform data analysis and data profiling to support application development and business requirements.

- Develop scalable and configurable data processing solutions using Scala and Apache Spark.

- Develop and maintain Spark DataFrame transformations using Scala/PySpark.

- Optimize application and Spark code for performance, scalability and configurability.

- Perform unit testing using automated testing/BDD frameworks.

- Participate in functional, integration, regression and UAT testing activities.

- Develop and execute automated regression test suites.

- Support automated deployment and release processes across environments.

- Troubleshoot production issues and provide technical support when required.

- Work collaboratively with cross-functional technology and data teams.

- Follow Agile/Scrum development practices, coding standards and engineering best practices.

- Analyze data lineage and profiling requirements and support data quality initiatives.

Required Skills

- **5+ years of hands-on experience in Scala development.**

- Strong hands-on experience with **Apache Spark**.

- Hands-on experience with **PySpark**.

- Minimum **2 years of experience working on HDFS-based development projects**.

- Strong SQL development and query-writing skills.

- Experience working with relational databases; **DB2 experience is preferred**.

- Strong understanding of Spark DataFrame transformations.

- Experience optimizing code and Spark applications for performance and scalability.

- Strong understanding of distributed data processing concepts.

- Experience working in Agile/Scrum development environments.

- Strong analytical and problem-solving abilities.

- Good oral and written communication skills.

- Ability to independently understand problem statements and translate them into technical solutions.

Scala Programming Expertise

Candidates should have a strong understanding of fundamental Scala concepts and be able to apply them practically during coding/problem-solving exercises.

Key areas include:

- Case Classes

- Higher-Order Functions

- Function Currying

- Partial Functions and Total/Complete Functions

- Call-by-Value vs. Call-by-Name

- Option, Some and None

- Functional programming concepts

- Collections and transformations

- Pattern matching

Spark Expertise

Candidates should be comfortable with hands-on Spark development and should be able to explain Spark execution from an architectural perspective.

Expected knowledge includes:

- Spark DataFrame transformations

- Transformations vs. Actions

- Jobs and Stages

- Spark execution model

- Narrow vs. Wide Transformations

- Shuffle operations

- Spark performance optimization

- Partitioning and parallelism

- Spark deployment modes

- Client Mode vs. Cluster Mode

- Distributed data processing concepts

Additional Skills

- Java exposure is an advantage.

- Experience with automated BDD/testing frameworks is an advantage.

- Experience with CI/CD and automated release processes is preferred.

- Knowledge of data lineage and data profiling is preferred.

- Experience supporting production applications is an advantage.

What We Are Looking For

We are specifically looking for candidates who are **hands-on Scala developers** and not candidates whose primary expertise is Java, Python or generic Big Data with limited Scala exposure.

Candidates should be able to:

- Write production-quality Scala code.

- Solve coding problems using core Scala concepts.

- Develop Spark DataFrame transformations.

- Explain how Spark code executes internally.

- Identify jobs, stages and transformations/actions.

- Optimize Spark applications for performance and scalability.

- Work independently with limited requirement clarification.

- Communicate technical solutions clearly to business and technology stakeholders.