Eternal Ltd. is an India-based technology company building a diversified ecosystem across food delivery, quick commerce, supply chain, and lifestyle services through Zomato, Blinkit, Hyperpure, and District, respectively. Its businesses span food delivery and dining, quick commerce, restaurant supply, and going-out experiences, serving millions of customers, merchants, and partners through technology-driven platforms.
Powered by technology and data, Eternal builds scalable products that enhance customer experiences, optimize operations, and enable smarter business decisions. As it continues to grow, Eternal remains focused on innovation, operational excellence, and sustainable long-term growth.
Eternal used self-hosted Apache Kafka® as its main logging platform to store its logs and then feed them to ClickHouse for logs management.
To avoid networking fees that can quickly stack up due to crossing interzone boundaries and from multiple levels of replication, Eternal kept their self-hosted Apache Kafka setup constrained to a single availability zone with one level of replication.
Knowing that costs were higher than they needed to be and that the use case (logging) was not latency sensitive, Eternal completed a proof of concept of WarpStream’s Bring Your Own Cloud (BYOC) to replace this self-hosted Apache Kafka setup with WarpStream.
Eternal created a POC WarpStream cluster and dual-wrote the production logging traffic to their existing Apache Kafka setup and WarpStream. The entire proof of concept only took two weeks and another two weeks to move to production.
The result? A 60% cost savings vs. self-hosted Apache Kafka running in a single availability zone and they no longer have to worry about replication levels (WarpStream does not need to replicate), interzone networking, or cloud disks by leveraging S3-compatible object storage.
Eternal’s workload is very temporal – it drops to 2.3 GiB/s throughput at the lowest point of the day (when people are sleeping and not ordering) – and spikes to 25 GiB/s throughput (across multiple clusters) at its peak. WarpStream allows them to auto-scale their cluster throughout the day and make sure it's always perfectly right-sized – something that was impossible on their prior, self-hosted Apache Kafka setup.
"By migrating from self-hosted Kafka to WarpStream for observability and logging, we've drastically reduced costs while maintaining reliability, thanks to quick auto-scaling and S3 storage with zero inter-AZ charges," said Rajat Taya, a Senior Engineering Manager with Eternal.
Eternal did not stop there as its Kafka architecture stretches beyond self-hosted Apache Kafka, and they wanted to see where else implementing WarpStream would produce benefits.
Managed Kafka was used for benchmarking – this is where Eternal’s backend services record live traffic requests into Kafka and then replay them later to measure performance and perform load testing.
Aside from costs due to cloud disks and interzone networking, a major issue with managed Kafka was cluster creation time; it can take as long as 20 to 30 minutes. Eternal replaced this managed Kafka cluster with WarpStream, which reduced cluster creation time to seconds and also enabled them to take advantage of WarpStream’s “scale to zero” functionality for periods where the cluster is idle.
“This is where WarpStream shines,” noted Taya.
Not only did Eternal simplify operations and eliminate these cluster-creation wait times, they achieved an 80% cost savings by replacing managed Kafka with WarpStream and were able to port their benchmarking – which averages 52 million messages per minute during a backfill – to WarpStream.
WarpStream was a perfect fit for Eternal’s latency-insensitive workloads referenced above, i.e., logging and analytics, and benchmarking. Eternal has some workloads that are more latency sensitive and for that, they leverage Confluent Cloud.
As you can see in the architecture diagram below, it’s not an either-or decision that Eternal had to make. They can treat Kafka topics like a service, deciding when data should flow to WarpStream or Confluent Cloud based on their cost and latency requirements. This enables them to achieve maximum cost savings and simplify operations while meeting their varying performance requirements on a workload-by-workload basis.
“Switching to WarpStream was a game changer. It made handling massive data streams effortless and freed our team from constant infrastructure headaches. Now, they can focus on more meaningful work. Plus, WarpStream's smooth Kafka integration made the whole transition painless and future-proofed our observability setup,” said Nishant Sarraff, a Software Development Engineer at Eternal.
.svg)
