Rockset versus Elasticsearch
for Real-Time Analytics

Elasticsearch was designed for log analytics where you want to monitor and search on immutable data. Rockset, in contrast, is designed for real-time analytics where relationships matter, data is constantly being updated and analyzed, and cloud-native efficiencies are a cost saver.

4x

Faster Ingestion

Rockset is designed for real-time ingest using Log Structured Merge Trees that are mutable at an individual field level. This enables Rockset to outperform Elasticsearch when dealing with high-velocity streaming data.

44%

Lower Infrastructure Costs

Rockset separates storage, ingest and query compute so you don’t need to overprovision resources for your workload. Furthermore, Rockset can save up to 100x on storage costs by using SQL-based rollups.

20x

Faster Development Time

Rockset is cloud-native, saving your team from needing to manage clusters, nodes, shards and indexes. Furthermore, Rockset’s Converged Index enables ad-hoc search and analytics without any index management so teams realize real-time analytics 20x faster.

SQL

Joins

Rockset supports full SQL, including joins, in an efficient way. In Elasticsearch, joins are not a first class citizen and they are prohibitively expensive so you’ll need to use workarounds that add complexity to your data model.

How Rockset Addresses Inefficiencies in Elasticsearch

Challenge #1:

Joins Are Expensive

Performing SQL-style joins is prohibitively expensive in Elasticsearch

Even Elastic, the company behind Elasticsearch, acknowledges that “performing SQL-style joins in a distributed system like Elasticsearch is prohibitively expensive.” As a workaround to the lack of SQL join support, you’ll need to either denormalize data, perform application-side joins or use nested objects or parent-child relationships. Each of these options adds complexity, including data duplication, managing data changes, manually tuning for performance and indexing compute and time.

Rockset supports full SQL joins, efficiently

Rockset stores data in a document data model while supporting full SQL, including joins. You can ingest deeply nested JSON and run sub-second joins with Rockset’s Converged Index, an index inspired by search indexes and column stores. Furthermore, Rockset’s massively distributed query execution engine adds to both scalability and speed.

Challenge #2:

Inefficient Real-Time Ingestion

Elasticsearch users batch documents to minimize the cost of frequent updates

Elastic recommends batching frequently changing fields to minimize the cost of document updates. That’s because when an update is made to a document in Elasticsearch, the old document is deleted and the new one is buffered and merged into a new segment. One user, with frequent inserts and updates, spent 70% of their CPU in Elasticsearch on merge operations.

Rockset supports in-place updates, inserts and deletes to avoid expensive reindexing

Rockset is designed to handle high-velocity streaming data. Under the hood, Rockset uses Log-Structured Merge Trees (LSM) to write data to any free location, bypassing I/O intensive read-modify-writes. Rockset is also a mutable database and applies inserts, updates and deletes at the individual field level, avoiding expensive reindexing of documents. As a result, Rockset can handle 10 TBs/second in a compute-efficient way.

Challenge #3:

Over-Provisioned Compute and Storage

Elasticsearch’s tightly coupled architecture leads to inefficient resource utilization

Elasticsearch has a tightly coupled architecture where the clusters contain the compute and storage for the workload and cannot be scaled independently, leading to inefficient resource utilization. Furthermore, because clusters are responsible for ingestion and queries, writes can interfere with reads and vice versa. At peak times, users have complained that compute contention caused their application to become unresponsive.

Rockset is cloud-native for better price-performance

Rockset is cloud-native and separates storage, query compute and ingest compute for better price-performance. Rockset avoids compute contention by scaling ingest and query compute independently. Furthermore, storage and compute are provisioned independently so you never need to overpay for resources you don’t use.

Challenge #4:

High Operational Burden

Even managed Elasticsearch requires in-depth knowledge to control for costs

Elasticsearch is a highly complex distributed database that requires hiring and retaining full-time engineers who are well-versed in Elasticsearch’s data management, query DSL, data processing and cluster management. One customer estimated a 6 month roadmap for their application on Elasticsearch and 3 full-time engineers to manage the system.

With Rockset you never manage clusters, nodes, shards or indexes

Rockset is cloud-native and you don’t manage indexes, nodes, clusters or shards. Rockset is optimized for hands-free operations while providing you with advanced compliance and production monitoring for visibility and control. You can scale Rockset up or down simply with a click of a button or an API call.

Demo Rockset

Elasticsearch may be powerful but it’s scary to manage. Explore Rockset for complex filtering, aggregations and search, all with the efficiency of the cloud.

Rockset:

100% cloud-native

Increase ingest speeds by 4x

Full SQL, including joins

Scale efficiently with Converged Indexing

Don’t fight Elasticsearch. Choose Rockset for Real-Time Analytics.

See Our Customers

Elasticsearch doesn’t support joins, so we were constantly denormalizing our data to get around this. It can take a week to set up a Spark job to denormalize each data set, and because of the data we deal with, we would experience significant space amplification due to denormalization.” Emmanuel Fuentes, Whatnot

Learn more