Python, Spark Scala Developer

Job Description:

• oin a high impact engineering team where you ll build and optimize large scale data processing solutions using Python Apache Spark and Scala
• This role is ideal for someone who enjoys solving complex data challenges improving performance and delivering reliable pipelines that power analytics and downstream applications
• You ll collaborate closely with data engineers platform teams and stakeholders to translate business needs into scalable implementations while promoting clean code testing discipline and continuous improvement
• If you re motivated by ownership enjoy mentoring others and want to work in a culture that values collaboration learning and measurable outcomes this opportunity offers the space to make a visible difference both through technical contributions and by raising the bar for engineering excellence

Key Responsibilities:

• Data Engineering Development
• Design develop and maintain scalable batch stream data pipelines using Spark with Scala and Python
• Implement efficient transformations joins aggregations and data quality checks for large datasets
• Build reusable frameworks utilities to standardize pipeline patterns and reduce delivery time
• Performance Reliability
• Tune Spark jobs partitioning caching shuffles memory executor settings to improve performance and cost efficiency
• Troubleshoot production issues perform root cause analysis and implement preventive fixes
• Ensure reliability through robust logging monitoring hooks and failure handling strategies
• Collaboration Delivery
• Work with stakeholders to refine requirements and deliver well documented production ready solutions
• Conduct code reviews enforce best practices and mentor team members on Spark Scala Python patterns
• Contribute to CI CD friendly development practices including testing version control and release readiness

Technical Requirements:

• Technology Analytics Packages Python Big Data Technology Big Data Data Processing Spark Technology Big Data Data Processing Scala

Additional Responsibilities:

• Bachelor s degree in Computer Science Engineering or a related field or equivalent practical experience
• 5 9 years of experience in data engineering or backend development with strong hands on delivery ownership
• Strong proficiency in Python and Apache Spark for large scale data processing
• Solid experience with Scala for Spark based development and production grade implementations
• Working knowledge of Hive and SQL based data querying processing concepts

Preferred Skills:

Technology->Big Data - Data Processing->Spark,Technology->Java->Apache->Scala,Technology->Analytics - Packages->Python - Big Data