• Performance and Scaling

Settings and Limits

This page covers the setting and limits placed on your Rockset organization which can be customized either by changing your Virtual Instance type or by contacting Rockset customer support. To view your current account settings and limits, navigate to the Settings tab of the Rockset Console.

Resource Allocations

The following settings are limits placed on your Rockset organization which can change based on your Virtual Instance category.

Note: The Paid category includes organizations using any dedicated Virtual Instance or the c12.shared Virtual Instance, while the Free category includes only organizations explicitly using the Free Virtual Instance type.

LimitsLimits in TrialLimits in FreeLimits in Paid
Storage Size1 TiB2 GiBUnlimited
Number of Collections3010Unlimited
Number of Collection Aliases3010Unlimited
Number of Query Lambdas303Unlimited
Number of Versions per Query Lambda5020Unlimited
Number of Unique Query Lambda Tags22Unlimited
Number of Users per Organization5050Unlimited

To change your Virtual Instance type, navigate to the Virtual Instances tab of the Rockset Console.

System Settings

The following settings are limits placed on your Rockset organization which may be customized for your specific use case by contacting Rockset customer support.

Peak Streaming Ingest Limit

This refers to the maximum rate at which you can add data into Rockset. It is measured as the number of uncompressed bytes of new data being added to your account per second.

The peak streaming ingest limit is dependent on your Virtual Instance type. On Shared Instances, the peak streaming ingest rate is limited to 1 MB/sec. Dedicated Instances do not have any hard limits on streaming ingest rate; however, your streaming ingest rate will be constrained by the Virtual Instance type that you have provisioned. Larger Dedicated Instances will have directly proportional lower data latencies due to the additional compute resources working on ingest. For instance, upgrading from a 2XLarge to a 4XLarge Dedicated Instance typically doubles your peak streaming ingest rate.

Note: The peak streaming ingest limit does not affect your bulk ingest limit during the initial bulk ingest mode which occurs immediately following collection creation. For instance, if you are creating a collection from MongoDB, DynamoDB or S3 and that source already has a large dataset, Rockset will automatically employ a bulk loading mechanism to index the data that is pre-existing in that source. This bulk loading process occurs immediately following the creation of the collection, and is not limited to the peak streaming ingest limit that is explained above. The bulk ingest rate is hundreds of gigabytes per hour. If you need a higher bulk ingest rate, please contact Rockset customer support.

After the bulk loading process is completed, Rockset streams new updates made to the source and applies those updates continuously. The peak streaming ingest rate limit described above now applies.

Data Size Limit

This refers to the total amount of data that you can store in Rockset.

Rockset accounts on the free tier have a data size limit of 2 GiB.

Rockset accounts on paid tiers do not have any data size limits; however, as a convenience to you, an artificial limit of 100 TiB is placed your account so that any runaway process on your account cannot inundate your account with data that you were not intending to store in Rockset. If you would like to raise this limit for your account, please contact Rockset customer support.

Query Memory Limit

This refers to the total amount of memory that a single query can use. This memory is used for storing transient data that is created by intermediate stages of your query execution.

On Shared Instances, each individual query has a memory limit of 100 MB which can be used during execution. On Dedicated Instances, the memory available for your queries are determined by the size of the Virtual Instance. In this case, the memory associated with your Virtual Instance is shared by all concurrent queries in your account (for example, larger Dedicated Instance sizes will have higher memory limits).

Concurrent Query Execution Limit

This refers to the maximum number of queries that you can execute concurrently in your account. This includes the queries you make directly from the Query Editor tab of the Rockset Console, Rockset API, or Query Lambdas.

On Shared Instances, you are limited to 2 concurrent queries at a time. Dedicated Instances have a default CQEL of 6 concurrent queries. This value can be increased to a level that provides the maximum number of concurrent queries while still maintaining an expected SLA. Beyond this limit, additional inbound queries will be queued for processing.