Senior Data Platform Engineer

Responsibilities:

• Lead and focus on scaling the data infrastructure, streamlining the data delivery, and designing and implementing AI automation to replace manual operations.
• Engage closely with a wide array of stakeholders, including executive, product, data, and design teams, providing high-level support for data infrastructure challenges, advising on technical data issues, and implementing self-serve solutions.
• Develop sophisticated AI-based analytical tools that integrate with the data pipeline to provide deep insights into key business metrics, including customer growth and operational efficiency.
• Architect and manage extensive, sophisticated data sets to meet complex business needs and requirements.
• Engage closely with a wide array of stakeholders, including executive, product, data, and design teams, providing high-level support for data infrastructure challenges and advising on technical data issues.
• Make a meaningful impact on the lives of our community members.
• Collaborate and mentor other senior engineers while providing thoughtful guidance using code, design, and architecture reviews.
• Contribute to defining technical direction, planning the roadmap, escalating issues, and synthesising feedback to ensure team success.
• Estimate and manage team project timelines and risks.
• Participate in hiring and onboarding for new team members.
• Lead cross-team engineering initiatives.
• Constantly lEarnIng about new technologies and industry standards in data engineering.

Requirements:

• 4+ years of experience in designing, building, and maintaining data infrastructure, and the ability to lead complex projects.
• Bachelor's, Master's, or PhD degree in computer science, computer engineering, or a related technical discipline, or equivalent industry experience.
• Deep knowledge of Kafka, Proficiency in programming languages like Python and Scala, and a good understanding of ML engineering.
• Strong knowledge of distributed computing frameworks, such as Apache Spark, with cloud platforms such as AWS, Azure, or GCP.
• Working experience with Databricks would be nice to have.
• Experience using AI-assisted development tools (e. g., GitHub Copilot, Cursor, ChatGPT, or similar tools) as part of your software development workflow.
• Deep understanding of database design, SQL, and NoSQL databases.
• Experience in managing large datasets and optimising database performance.
• Proficiency in Git and Terraform, and experience in deploying continuous integration and continuous deployment (CI/CD) practices.
• Experience in managing event-driven systems, particularly with Kafka in cloud environments.
• Expertise in developing and implementing data governance frameworks, policies, and procedures to ensure data quality, compliance, and effective data management practices.
• Deep understanding of data security principles, including encryption, decryption, and secure data storage and transfer protocols.