Data Engineer
Reelo Technologies · Bengaluru · onsite · Posted 1d ago
Undisclosed2–4 yrsfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
Job Summary
Responsibilities
1. Data pipelines
2. Data accuracy and reconciliation
3. Moving to incremental processing
4. ClickHouse optimisation
Must have
Nice to have
Our stack
What success looks like
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
We're hiring a Data Engineer to own our data platform end to end: the pipelines that move data from MongoDB into ClickHouse, the accuracy of every number our customers see, and the performance and cost of ClickHouse itself.
You'll work on our live production stack and use AI-assisted development (Claude Code) as part of your everyday workflow. You'll be the first responder when something in your area breaks, so you'll care about root causes, not restarts.
Responsibilities
1. Data pipelines
- Run and improve the event flow from MongoDB through Kinesis, SQS and Lambda consumers, plus ClickPipes, into ClickHouse.
- Keep consumers healthy: retries, batch item failures, dedup, dead-letter queues, and Mongo and Redis connection limits.
- Fix failures quickly and find the root cause. A restart doesn't count as a fix.
- Keep dev, staging and prod schemas in sync, and handle DDL and schema initialisation safely.
2. Data accuracy and reconciliation
- Own reconciliation between MongoDB (source) and ClickHouse (reports) for orders, feedbacks, loyalty points, credits and comm reports.
- Investigate customer-reported mismatches (points balances, sales totals, undelivered counts, segment counts) and close each one with a root cause.
- Build checks that catch drift before customers do: daily row counts, sums and key metrics compared per brand.
- Get timezone, order_type and status handling right everywhere a metric is computed.
3. Moving to incremental processing
- Replace full recompute on every event with incremental updates, using ClickHouse materialized views and AggregatingMergeTree instead of repeated MongoDB aggregations.
- Ship migrations in small, switchable steps, each verified against the old path before cut-over.
- Retire legacy pipeline paths (Kinesis Lambda ClickHouse) once the replacements are proven correct.
4. ClickHouse optimisation
- Own schema design: ORDER BY, partition keys, granularity and engine choice, following ClickHouse best practices.
- Tune the slowest and most expensive queries weekly using system.query_log, and keep the ClickHouse Cloud bill within target.
- Prevent merge pressure: no forced whole-partition merges, sensible insert batching, healthy part counts.
- Make sure dashboard and export endpoints meet their latency targets.
On-call: Each of these four areas carries its own on-call responsibility. When it breaks, you're the first responder.
Must have
- Strong ClickHouse and analytical SQL (CTEs, window functions, argMax, joins)
- Production experience running SQS or Kinesis pipelines in Node.js
- Solid MongoDB experience (aggregations, indexes, change data)
- Hands-on use of Claude Code or a similar AI coding tool, with the discipline to verify AI output through tests and real data before shipping
Nice to have
- CDC tools such as PeerDB or Debezium
- EMR Serverless or Spark
- Python for data checks
- Experience moving batch recompute to incremental processing
Our stack
- AI-assisted development: Claude Code
- Pipelines: AWS SQS, Kinesis Data Streams, Lambda, Step Functions
- Analytics database: ClickHouse Cloud, ClickPipes (CDC)
- Source database: MongoDB (Mongoose)
- Compute and deploy: AWS ECS Fargate, Serverless Framework, EventBridge Scheduler, S3, SSM
- Caching and queues: Redis, Bull
- Language: Node.js (JavaScript)
- Observability: Datadog, AWS CloudWatch
What success looks like
- Customer-reported data mismatches are rare, and when they happen they're closed with a root cause.
- Drift is caught by your checks before customers notice.
- Metrics run on incremental paths, and the old pipelines are retired.
- Dashboards and exports hit their latency targets, and the ClickHouse bill stays on target.
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
Sourced from naukri
Skills
CdcAutomationDatadogData Engineerclaudecachingamazon web servicesAws EmrClaude CodeeventbridgeDashboardAnalyticsClickHouseAWS CloudWatchAWS ECSData AnalyticscloudCustomer AcquisitionCustomer RelationshipsData PipelineArtificial Intelligence
About Reelo Technologies
Reelo Technologies
Software Product
2016
10 years old
ahmadabad, gujarat
India
₹5.9L PA avg
Avg at Reelo Technologies
About
Reelo makes customer retention engagement easy for every retail business. The world is rapidly moving towards personalization and customers are rapidly moving to digital, faster than ever before. The way consumers make purchasing decisions has dramatically changed and retailers are struggling to adapt to this new reality. The current tools in the market are far too overpowered, complex and expensive that they become inaccessible for most small medium sized retailers. These tools are meant to be used by marketing teams of large enterprise brands.
Industries
Software Product
Employee ratings
10 reviews
3.0/ 5
Culture
3.2
Career growth
2.3
Work-life
3.3
Employees rate it well for
Work-Life Balance31Work Satisfaction31
Find them on