Lead Software Engineer

Responsibilities:

• Focus on major areas of work, typically 20% or more of role % of Time.
• Be a key player in a cross-functional team of engineers who build software for a large-scale data processing ecosystem, supporting real-time and batch data pipelines for analytics, data science, and operations.
• Maintain software engineering and architecture best practices and standards within the team and wider organisation, along with a culture of quality, innovation, and experimentation.
• Evangelise and evolve the platform, best practices, data driven decisions; identify new use cases and features and drive adoption.
• Be a part of the magic (partner) for countless teams as they use the Data Lake platform, helping architect and solution for success.
• Work together with other engineering teams to influence them on system design and optimisation.
• Collaborate efficiently with Product Managers and other developers to build datastores as a service.

Requirements:

• Minimum and Preferred. Inclusive of Licenses/Certs (include functional experience as well as behavioural attributes and/or leadership capabilities).
• Experience deploying and running cloud-based applications (preferably AWS), utilizing infrastructure- as-code frameworks like Terraform.
• Experience with Python, Spark, Airflow, Kafka, Iceberg, or Delta Lake.
• Ability to dive deep into technical components; swiftly understand end-to-end workflow and the overall systems.
• Ability to design the architecture of CICD, automation, and big data systems, elaborating existing systems with the latest technology.
• Care deeply about craftsmanship in your software, and can work backwards from the customer experience.
• Excellent written and verbal communication skills.
• Knowledge of Data Engineering practices and tools, such as Spark, Snowflake, ETL concepts, etc.
• 7+ years of relevant professional experience and a Bachelor's degree in Computer Science or related field OR 5+ years of relevant professional experience and a Master's degree in Computer Science or related field.