WarpStream is a Kafka compatible data streaming platform built directly on top of S3: no inter-AZ bandwidth costs, no disks to manage, and infinitely scalable, all within your VPC.
.svg)
Agents stream data directly to and from object storage with no buffering on local disks and no data tiering.
Create new serverless “Virtual Clusters” in our control plane instantly.
Support different environments, teams, or projects without managing any dedicated infrastructure.
Agents stream data directly to and from object storage directly with no buffering on local disks and no data tiering.
Create new serverless “Virtual Clusters” in our control plane instantly.
Support different environments, teams, or projects without managing any dedicated infrastructure.
.svg)
.svg)
WarpStream is protocol compatible with Apache Kafka® and Kinesis, so you can keep using all your favorite tools and software.
No need to rewrite your application or use a proprietary SDK. Just change the URL in your favorite Kafka client library and start streaming!
.svg)
.png)
.png)
.png)
Don't see an answer to your question? Check our docs, or contact us directly.
The real world is complex and actual cost savings will vary from workload to workload. However, unlike other data streaming systems, WarpStream was designed from the ground up around cloud economics.
WarpStream's unique architecture of leveraging object storage directly with no local disks or manual replication avoids almost all interzone bandwidth costs, and ensures that data is always stored in the the most cost effective way possible.
Our hybrid deployment model means that raw data never leaves your environment.
The only data we receive is the high level metadata required to operate your control plane like topic names and partition counts.
Support for the full Kafka protocol is still a work in progress for us, but broadly it means that you should be able to take existing software like Spark, Flink, or custom applications that embed a Kafka client, change the Kafka URL to point at WarpStream, and have everything continue working as intended.
More specifically, this means that we support topics, partitions, consumer groups, etc. We also maintain all the same per-partition ordering guarantees that Kafka does, and even take things a step further by ensuring that producing data to any set of topics/partitions is fully atomic by default.
The difference is subtle, but makes all the difference.
Systems like Kafka that do traditional "data tiering" can reduce storage costs, but remain highly stateful and must be operated with extreme care. In addition, data tiering based systems do not solve the interzone networking problem and thus remain cost prohibitive for high volume workloads.
WarpStream's approach is not data tiering. WarpStream uses S3 as the primary and only storage for all streaming data. This brings all the traditional benefits of data tiering to the system, but also makes the WarpStream agents completely stateless, which makes operating them trivial. In addition, it avoids all interzone networking fees, which dramatically reduces the cost of high volume workloads.
See our documentation for more details.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
.svg)
.png)
