Software Development Engineer II

Job Summary

We are looking for an experienced Data Engineering Developer (SDE2) to design and build scalable, high-performance data solutions. The ideal candidate will have strong expertise in Apache Spark, Databricks, and Scala, with a passion for building reliable data pipelines and ensuring data quality at scale. If you enjoy solving complex data challenges and working in a fast-paced, collaborative environment, we would love to hear from you.

Responsibilities

• Design, develop, and maintain scalable and reliable ETL pipelines using Apache Spark and Databricks.
• Write high-performance, production-grade Scala code for data processing and transformation.
• Implement and manage observability, monitoring, and alerting frameworks to ensure system health and data integrity.
• Develop and maintain CI/CD pipelines for data applications and infrastructure using tools such as GitHub Actions or Azure DevOps.
• Ensure adherence to data governance, data quality, and security best practices across all data systems.
• Collaborate closely with cross-functional teams including platform, analytics, and product teams to deliver robust data solutions.
• Continuously improve system performance, scalability, and operational excellence through best engineering practices.
• Stay updated with emerging technologies, including advancements in Generative AI (GenAI), and evaluate their applicability to data platforms.

Requirements

• Strong hands-on expertise in Apache Spark and Databricks.
• Proficiency in Scala, with a solid understanding of functional programming concepts.
• Experience in building and maintaining scalable ETL/data pipelines.
• Hands-on experience with CI/CD tools and practices (e.g., GitHub Actions, Azure DevOps).
• Experience in implementing monitoring, logging, and alerting systems for distributed data platforms.
• Good understanding of data governance, security, and data quality principles.
• Strong analytical, problem-solving, and debugging skills.
• Ability to work effectively in a collaborative, Agile environment.
• Nice-to-Have: Exposure to Generative AI (GenAI) or AI/ML data pipelines.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.