SDE - 2 - Data Engineering

We're looking for a passionate Sr. Software Engineer (SDE-2) - Data Engineer to develop scalable data pipelines and analytics solutions. You'll solve complex data challenges, optimize systems for performance, and collaborate with teams to drive data-driven innovation that fuels our growth.

Responsibilities

• Contribute to the design and architecture of Pluang's Data Platform, leveraging Apache Spark, HDFS, Kafka Streams, and ETL pipelines to sync data to the Data Lake and Data Warehouse.
• Optimize data mart ETLs to make them scalable and efficient, collaborating with the Data Analytics teams to ensure business insights are delivered timely and accurately.
• Build frameworks/infrastructure in GCP that enable data analysts to be able to deploy reporting more efficiently.
• Build/maintain a data catalog platform to capture all the metadata and documentation of Pluang's vast datasets.
• Contribute to the foundation infrastructure for implementing AI agents to maximize business value.
• Implement monitoring and alerting frameworks to ensure the scalability, availability, and performance of the Data Platform, driving Pluang's data-driven future.

Requirements

• Bachelor's in Computer Science with expertise in Apache Spark, Apache Kafka, Google BigQuery, and serverless tech like AWS Lambda and GCP Cloud Functions.
• Proficient in Python/Scala, building ETL pipelines with Apache Airflow/dbt, and working with data streaming technologies for real-time processing.
• Strong analytical and problem-solving skills, with excellent communication abilities to collaborate across teams.
• Fintech experience is a plus, with an understanding of the unique challenges in data engineering for the industry. This job was posted by Vibhuti Juneja from Pluang.