Rockset vs Snowflake

Compare and contrast Rockset and Snowflake by architecture, ingestion, queries, performance, and scalability.

Rockset vs Snowflake Architecture

Architecture

Rockset

Snowflake

Deployment model

SaaS - infrastructure, software and cluster ops managed by service provider

Use of storage hierarchy

• SSDs store shared hot data, accessible from any Virtual Instance cluster • Cloud storage for durability

Cloud object storage for shared data accessible from any virtual warehouse

Isolation of ingest and query

Yes - separate compute clusters (Virtual Instances) for ingest and query

Yes - separate virtual warehouses for batch data loading, ELT jobs and queries

Separation of compute and storage

Yes

Isolation for multiple applications

Yes - separate compute cluster (Virtual Instance) for each application

Yes - separate virtual warehouses for each workload

Rockset is built to be a cloud-only database and does not have a self-managed option. It disaggregates compute from both hot storage and cloud storage, allowing multiple isolated compute clusters to run on the same shared data.

Snowflake is the data warehouse built for the cloud. Snowflake is well-known for separating storage and compute for better price performance. With Snowflake, multiple virtual warehouses can be spun up or down for batch data loading, transformations and queries all on the same shared data.

Rockset vs Snowflake Ingestion

Ingestion

Rockset

Snowflake

Data sources

Managed data connectors to: • Events streams (e.g. Kafka, Kinesis) • Database CDC (e.g. MongoDB, DynamoDB, MySQL, PostgreSQL) • Data lakes (e.g. S3, Google Cloud Storage)

• Third party ETL tool to ingest data into Snowflake including Fivetran, Hevo or Striim • Bulk loading from S3, GCS, Azure Blob Storage • Sink Connector for Apache Kafka in Confluent Cloud

Semi structured data

• Ingests JSON and XML without a predefined schema • Ingests nested data

Ingests JSON and XML as a VARIANT data type

Transformations and rollups

Yes - using SQL ingest transformations

• Third party ELT/ETL tools like dbt • Simple COPY commands at data loading for column recording, omission and casts

Rockset has built-in connectors that manage streaming ingestion from common data sources. It has native support for semi-structured data, so that nested JSON and XML can be ingested and queried as is.

Snowflake is an immutable data warehouse that is built for batch ingestion and relies heavily on the modern data stack ecosystem for data connectors and transformations. Snowflake has a number of integrations to ETL and ELT solutions including Fivetran, Hevo, Striim and dbt. While Snowflake does have support for semi-structured data in the form of a VARIANT type, it is best to structure the data for optimal query performance.

Rockset vs Snowflake Performance

Performance

Rockset

Snowflake

Updates

Documents are mutable and can be updated at the field level

Data warehouse with immutable storage. Updates rewrite and merge entire partitions

Indexing

• Converged Index (row, columnar and inverted index) built on all data by default • All queries are resolved through the index

Query latency

50-1000ms queries on 100s of TB

Seconds to minutes on petabytes of data

Storage format

Converged Index, comprising a rowstore, columnstore and inverted index

Compressed columnar format stored in cloud object storage

Streaming ingest

• Ingests on a per-record basis • Data latency is typically 1-2 seconds

• Ingests on a batch basis • Snowpipe typically ingests in minutes

Rockset is designed to make streaming data queryable as quickly as possible by avoiding the need to batch data. It also updates documents efficiently by only reindexing fields that are part of an update request. Rockset indexes all data by default, which results in storage amplification but also enables low-latency queries that require less compute.

Snowflake is designed for batch analytics with analysts and data scientists infrequently accessing large-scale data for trend analysis. Snowflake, like many data warehouses, is immutable and does not support frequently changing data efficiently. Snowflake uses a columnar store to return aggregations and metrics efficiently, often with query response times in the seconds to minutes on petabytes of data.

Rockset vs Snowflake Queries

Queries

Rockset

Snowflake

Joins

Yes

Query language

SQL - with extensions for nested data

SQL

Developer tooling

• Data APIs - saved SQL queries executed via REST endpoint • Python, Java, Node.js and Go SDKs • UDFs

• SQL APIs - make SQL calls to Snowflake programmatically • UDFs for Javascript, Python, Java and SQL functions • Go, JDBC, .NET, Node.js, ODBC, PHP, Python drivers

Visualization tools

Integrations with Tableau, Looker, Grafana, Superset, Power BI, Retoolwhic

Integrations with QuickSight, Chartio, Domo, Looker, PowerBI, Mode, Qlik, Sigma, Sisense, Tableau, ThoughtSpot and more

Rockset supports SQL as its native query language and can perform SQL joins. Users can create data APIs by storing SQL queries in Rockset that are executed from dedicated REST endpoints. Rockset integrates with some common visualization tools, but BI is not Rockset’s primary use case.

Snowflake supports SQL as its native query language and can perform SQL joins. Snowflake for developers introduced a number of developer tools including SQL APIs, UDFs and drivers to support application development. As Snowflake was originally built for business intelligence workloads, it integrates with a number of visualization tools for trend analysis.

Rockset vs Snowflake Scalability

Scalability

Rockset

Snowflake

Vertical scaling

Resize compute clusters (Virtual Instances) via API or in the console

Resize virtual warehouses via web interface or using DDL commands for warehouses

Horizontal scaling

• Add or remove compute clusters (Virtual Instances) via API or in the console • Scale out compute clusters for higher concurrency • Use separate compute clusters to isolate ingest from query or for multiple isolated applications • No rebalancing required

• Multi-cluster warehouses allocate additional clusters for higher concurrency workloads • Auto scaling policies can be set

Rockset Virtual Instances are distributed compute clusters that can be scaled up for faster queries or scaled out for practically unlimited concurrency or if compute isolation is needed. Rockset has shared storage that scales automatically and independently, so no rebalancing is required.

Snowflake virtual warehouses can be scaled up for faster queries or scaled out using multi-cluster warehouses to support higher concurrency workloads. Snowflake has shared blob storage that scales automatically and independently.