Senior Software Engineer - Confluent Kafka Team

Introduction

At IBM Software, we transform client challenges into solutions. Building the world's leading AI-powered, cloud-native products that shape the future of business and society. Our legacy of innovation creates endless opportunities for IBMers to learn, grow, and make an impact on a global scale. Working in Software means joining a team fueled by curiosity and collaboration. You'll work with diverse technologies, partners, and industries to design, develop, and deliver solutions that power digital transformation. With a culture that values innovation, growth, and continuous learning, IBM Software places you at the heart of IBM's product and technology landscape. Here, you'll have the tools and opportunities to advance your career while creating software that changes the world.

Your Role And Responsibilities

About the Role:

We are looking for a Senior Software Engineer to join the [1] Tableflow team. In this role, you will build the critical components that turn Confluent Cloud data streams into open table formats. You will bridge the gap between operational streaming (Kafka) and analytical data lakes (Apache Iceberg, Delta Lake) by building the control plane and data plane components that materialize topics into tables in real-time.

About The Team

We are a team of passionate engineers who love solving complex distributed systems and infrastructure problems. The Tableflow team creates the intelligent connective tissue between Confluent's data streaming platform and the modern data lakehouse. The team's work focuses on two core pillars:

Control Plane: Building the robust orchestration layer that manages the entire lifecycle of tables across AWS, Azure, and GCP.

Open Table Formats and Ecosystem: Building the translation logic to publish Kafka data as Tables (Apache Iceberg and Delta Lake) and also integrates with the Lakehouse ecosystem (AWS Glue, Unity Catalog).

What You Will Do

Design and implement data plane components that publish data into open-table formats and integrate with the ecosystem.

Develop control plane services and APIs that orchestrate table lifecycles and resource management.

Contribute to relevant open-Source projects such as Apache Iceberg.

Own code quality, test coverage, documentation, and maintainability for your components.

Drive technical designs, leading discussions to curate and moderate decisions with stakeholders.

Collaborate cross-functionally with Product, Design, and other engineering teams to deliver complex features.

Lead operational excellence by resolving high-severity incidents, building observability tools, and designing automated workflows to ensure 99.99% availability.

Be an active team player by leading meetings, taking initiative, and participating in on-call rotations.

References

Visible links

• https://www.confluent.io/product/tableflow/
  
  

Preferred Education

Master's Degree

Required Technical And Professional Expertise

• A degree in Computer Science, or equivalent experience.
• 3+ years of hands-on software engineering experience.
• Hands-on experience building services on public clouds (AWS, Azure, or GCP).
• Self starter who is able to quickly ramp up on the important bits of our overall product.
• Good understanding of distributed systems design and development.
• Experience running production services, debugging complex integration issues, and driving post-mortems.
• Ability to move fast and iterate with a sense of urgency while delivering results without surprises.
• A growth mindset and the ability to collaborate effectively with team members and take on customer problems.
  
  

Preferred Technical And Professional Experience

• Experience using and/or working on data infrastructure such as SQL databases, Apache Kafka, Apache Flink or Apache Spark.
• Hands-on experience with Open-Table Formats (Apache Iceberg, or Delta Lake) and Data Catalogs (Unity Catalog or AWS Glue).
• Interest in evangelism: Giving talks at tech conferences, writing blog posts evangelizing your work