Data Engineer - Apache Spark | Kafka | Flink | Trino

Job Summary


Responsibilities

• Work and enhance big data related opensource technologies.
• Build and operate Kafka-based streaming applications, including ingestion, filtering, enrichment, and replication use cases.
• Develop data processing jobs using Apache Spark and/or Apache Flink, following established patterns and platform guidelines.
• Work with data lake technologies (Apache Iceberg) to manage large analytical datasets.
• Tune jobs for performance, scalability, and cost efficiency.
• Write clean, testable code and participate actively in code reviews.

Minimum Qualifications

• At least 4-8 years of experience in data engineering and software development.
• Ability to work with opensource ecosystem.
• Ability to write high-quality code in Java/Scala, Python, or equivalent languages.
• Practical experience with Apache Spark and Apache Flink.
• Practical experience with query engines like Trino/Presto.
• Experience working with Kafka and scaling for high volume workloads.
• Strong SQL skills and understanding of data modeling for analytical workloads

Preferred Qualifications

• Hands-on experience with real-time analytics stores like Apache Pinot/Clickhouse for low-latency analytical queries.
• Familiarity using AI Tools for debugging and development (CoPilot, Cursor, Codex).
• Familiarity with data lake table formats such as Apache Iceberg.
• Familiarity with data governance tools like DataHub.
• Experience working with tools like DolphinScheduler or Airflow.
• Experience working with Lakehouse management systems like Apache Amoro.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.