Virtual Instances

A Virtual Instance is a set of compute resources used to process streaming ingest and queries.

Every Rockset organization starts with one Virtual Instance per active region. This becomes the default Virtual Instance for that region. Unless otherwise explicitly specified, all streaming ingest and queries are routed to this Virtual Instance.

Virtual Instances are compute resources and are separate from data storage. Virtual Instances can have mounted collections. Mounting a collection allows the Virtual Instance to serve queries that access that collection. The same collection can be mounted on any number of Virtual Instances.

You can manage your account's Virtual Instances in the Virtual Instances tab of the Rockset Console.

Virtual Instances can be hosted in one or more regions. For a list of current supported regions refer to the list here. Looking for a different region? Contact us at support@rockset.com.

Choosing a Virtual Instance

Virtual Instances come in two types - Shared and Dedicated.

Shared Virtual Instances share a global resource pool with other accounts and are a cost-effective way to develop and prototype. There are different sizes of Shared Virtual Instances, which have limits listed here.

Dedicated Virtual Instances are isolated and meant for production use cases. Rockset offers the following Dedicated Virtual Instance sizes:

Note: Ingest represents the peak streaming ingest limit for insert-only workloads with record sizes of 1 KiB. Source and collection configurations such as average document size, ingest transformations, and clustering may lead to different observed throughputs.

Typically, the compute you'll require will depend on four things:

• data size: the size of the data you're querying
• query complexity: the complexity of your query
• query load: the number of concurrent queries
• query latency: the query latency requirement of your application
• streaming ingest load: the number and size of documents being processed, transformed, and indexed from the source

You can choose a Virtual Instance size that meets your needs.

Multiple Virtual Instances

Note: The ability to create additional Virtual Instances is currently in Public Beta. Data refresh intervals are limited to 1 hour or more. Contact us for early access to live data on secondary Virtual Instances.

A Virtual Instance is a set of compute resources used to process streaming ingest and queries.

Every Rockset organization starts with one Virtual Instance per active region. This becomes the default Virtual Instance for that region. Unless otherwise explicitly specified, all streaming ingest and queries are routed to this Virtual Instance.

Virtual Instances come in two types - Shared and Dedicated. Shared Virtual Instances share a global resource pool with other accounts and are a cost-effective way to develop and prototype. Dedicated Virtual Instances are isolated and meant for production use cases.

Virtual Instances are compute resources and are separate from data storage. Virtual Instances can mount collections. Mounting a collection allows the Virtual Instance to serve queries that access that collection. The same collection can be mounted on any number of Virtual Instances.

Why Virtual Instances?

Virtual Instances allow you to isolate and independently scale compute for streaming ingest, low latency queries, and multiple applications also referred to as compute-compute separation. The workload served by one Virtual Instance does not affect the workload running on any other Virtual Instance.

Streaming Ingest Isolation & Scaling

Virtual Instances allow you to isolate and independently scale compute for ingest and queries. To eliminate the problem of compute contention, we recommend using one Virtual Instance for handling all the streaming ingest workload exclusively (with no query requests routed to that Virtual Instance, sometimes referred to as an “Ingest VI”), and additional VIs for each query use case.

Note: Today only the default Virtual Instance may be used for streaming ingest. We recommend dedicating this Virtual Instances to streaming ingest and using other Virtual Instances for queries.

Isolation & Scaling for Multiple Real-time Applications

Suppose you're using Rockset to power both a feature in your customer-facing application and an internal dashboard. A spike in internal usage shouldn't degrade the performance of your customer-facing application, and you should be able to isolate and scale compute for each application independently (upsizing customer-facing compute for an anticipated event, for example). We recommend using a dedicated Virtual Instance for each application.

Concurrency Scaling

Applications with fluctuations in demand can provide a consistent and predictable performance by scaling out the query requests to multiple virtual instances. Each of the virtual instances can mount the same set of collections and serve the same queries, allowing for better load balancing of increased loads during peak times.

Isolation for Analysis/Development

Virtual Instances allow you to isolate compute for one-off data analysis sessions (say running a set of complex compute-intensive aggregations for a monthly or quarterly report). You can run a compute-intensive query on one Virtual Instance while your application queries are safely served from another.

Likewise, you can use a dedicated Virtual Instance to develop or test new queries for new features without affecting your production workloads.

Multi-Tenant Isolation

Virtual Instances allow you to provide your own tenants (whether they be internal teams or customers) compute isolation and track usage and spend accordingly. You can mount collections across these Virtual Instances to share data as needed while using custom roles to limit access.

Providing a dedicated Virtual Instance for your largest tenants enables a better experience by avoiding resource contention from your smaller tenants and vice versa.

Using Virtual Instances

Streaming Ingest

Today the default Virtual Instance is set to process streaming ingest for all collections. You cannot configure this behavior otherwise at this time.

Collection Mounts: Making Data Available for Queries

A collection mount represents a relationship between a collection (storage) and a Virtual Instance (compute). Queries routed to a particular Virtual Instance can only access a given collection if that collection is mounted to the Virtual Instance. Collection mounts do not incur any additional storage costs, but they can incur additional CPU overhead on the Virtual Instance to which they are mounted.

Mounting and unmounting a collection should only take a few seconds, but sometimes it can take longer (up to one minute). You can manage collection mounts through the Rockset Console or directly through the API.

There are two types of data refresh intervals: Periodic and Continuous

Continuous refresh means that mounts stay up-to-date with their underlying collections in real time. Any updates to the underlying collection are immediately reflected in the mount and in query results issued to the respective Virtual Instance.

Additionally:

• Continuous refreshes will incur a small CPU overhead required to process streaming updates to your underlying collections.
• When you create a new collection, that collection will automatically use continuous refresh with your default Virtual Instance.

Note: For the Public Beta, continuous refresh cannot be created outside your default Virtual Instance. It is available in early access for all Virtual Instances.

Periodic refresh to mounts means that they do not receive real-time updates from the underlying collection. These mounts only reflect the state of the collection as of the time when the collection was mounted on the Virtual Instance, and get the updated data from the collection during each refresh period. The collection can continue to receive updates via the default Virtual Instance but these new updates won't be visible to the queries running on Virtual Instance until the next refresh period.

Additionally:

• When picking a 'never' period (e.g. no refresh) the mount will expire after 24 hours. After the 24-hour time period expires, the collection is automatically unmounted.
• Periodic refreshes do not induce any ongoing CPU overhead, since they don't process any collection updates. However, some CPU overhead is incurred during the refresh and mounting processes.
• We do not support multiple, simultaneous mounts of the same collection to the same Virtual Instance.
• You can only choose one refresh period for all collections mounted to a given Virtual Instance. This means you can't pick different refresh periods for different collection mounts on the same Virtual Instance.
• You can mount and unmount collections through the 'Virtual Instances Details' page in the Console or directly via the API. If you try to query an unmounted collection through the Query Editor, you'll see an option to mount the required collection(s). Attempting to query unmounted collections via the API will return an error.
• To view when the last refresh was completed go to the Mounted Collections tab by clicking the relevant Virtual Instance in the Virtual Instances tab.

Routing Queries to Virtual Instances

In the Console Query Editor, you can select the Virtual Instance you're sending queries to:

You can also specify the Virtual Instance for queries through the API.

You can find the virtualInstanceId you need to use in the Virtual Instance details page with a copy button right next to it:

Copy the above ID use it with the query endpoint, as follows:

https://api.[region].rockset.com/v1/orgs/self/virtualinstances/[virtualInstanceId]/queries

Alternatively use the request body for executing Query Lambdas and include the virtualInstanceId field.

If no Virtual Instance is specified, the query will be routed to the default Virtual Instance.

Virtual Instance Auto-Scaling

Note: Virtual Instance Auto-Scaling is currently in Private Beta. Contact us for early access to this feature.

CPU-based vertical Auto-Scaling is a technique that automatically adjusts Virtual Instance size according to current workload demands. It is the most cost-efficient way to run Rockset while maximizing performance. With this feature enabled, the Virtual Instance size will auto-scale up during heavier workloads to deliver better performance and auto-scale down when idle to avoid over provisioning resources and reduce costs.

How Rockset Auto-Scales Virtual Instance Size

Rockset analyzes Virtual Instance CPU utilization to assess whether to auto-scale a Virtual Instance size up or down.

• Auto-Scale Up occurs when maximum CPU utilization exceeds 75% for 3x consecutive durations of 5 minutes.
• Auto-Scale Down occurs when average CPU utilization is below 25% and never exceeds 40% for 24x consecutive durations of 5 minutes.

Configuring an Auto-Scaling Policy

Auto-Scaling can be configured in the Rockset Console or through the Rockset API.

To create an Auto-Scaling policy on a Virtual Instance, go to the Virtual Instance tab of the Rockset Console and navigate to the Edit Virtual Instance menu for the Virtual Instance you want to configure. Define the minimum and maximum sizes you will allow the policy to scale the Virtual Instance.

Note: Virtual Instance Auto-Scaling is currently only available for Default Virtual Instances.

Monitoring Auto-Scaling Events

When an Auto-Scaling occurs:

• Rockset logs Auto-Scaling events in the Activity Logs in the Virtual Instance details page.
• Rockset sends an email with details about the Auto-Scaling action.

Monitor how Rockset makes Auto-Scaling decisions based on CPU utilization with Virtual Instance metrics monitoring.

Managing Virtual Instances

Suspending and Resuming

Virtual Instances can be suspended and resumed. Suspending a VI will unmount all collections. Collections will not be automatically remounted when the Virtual Instance is resumed.

You can configure an Auto-Suspend Policy for a Virtual Instance. The Virtual Instance will suspend itself if it does not receive new queries during this duration.

Note: The default Virtual Instance cannot be suspended since it needs to process incoming data at all times.

Monitoring

You can find metrics specific to each Virtual Instance in the Console. The main Metrics tab has a 'Virtual Instance' selector, and each Virtual Instance Details page will also include some metrics.

The Metrics Endpoint will include Virtual Instance as a label in all compute-related metrics by default - allowing you to monitor each Virtual Instance independently as needed.

Access Management

You can scope privileges to specific Virtual Instances to restrict access to them. The following privileges can be given:

• Create: allows creating new Virtual Instances
• Query: allows sending queries to a particular VI
• Update: allows upsizing and downsizing as well as mounting/unmounting collections for a particular VI
• Suspend/Resume: allows suspending and resuming a particular VI
• Delete: allows deletion

You can find the full Role-Based Access Control reference here.

FAQs

Is the Shared Virtual Instance suitable for production workloads?

The Shared Virtual Instance is not suitable for production workloads. Accounts in this Virtual Instance share a global resource pool and isolation is not guaranteed. Your data is strictly kept isolated from other accounts but you share CPU and memory resources with other accounts. Sharing resources with other accounts allows you to utilize a lot of resources at a cost-effective price and is a great place to start using Rockset. A noisy neighbor can affect the latency and throughput of your queries. Bursty workloads from accounts may impact your query performance and availability.

The Shared Virtual Instance is usually useful for development and prototyping.

Will moving to a larger Virtual Instance make my queries faster?

Generally, adding compute by switching to a larger Virtual Instance will make your queries faster. Typically, doubling CPU allocation (e.g., moving from Medium (8 CPUs) to Large (16 CPUs) will approximately double your query performance (e.g., reduce query latency from 100ms to 50ms). Note that eventually your queries will not be bound by compute and instead reach the limitations of the underlying infrastructure — at which point adding additional compute will cease to affect latency.

If I expect my query load to increase significantly, do I need to move to a larger Virtual Instance?

Generally, yes. For example, if you're running 50 queries-per-sec and add an additional 50 queries-per-second (for 100 queries-per-second total), each query will get a proportionally smaller compute allocation, and query latency will increase accordingly (generally proportional to the increase in QPS). In this example, you might notice a query latency increase of approximately 2x. If you want your query latencies to remain the same even when you double your QPS, you should migrate to a Virtual Instance that is 2x larger in size.

Alternatively, you can choose to create additional Virtual Instances, mount the required collections, and spread the query request load among those Virtual Instances. You can suspend and resume those Virtual Instances based on when the load is higher or lower.

We always recommend stress-testing production workloads before launching your application on Rockset.

Does a Virtual Instance have a fixed amount of compute resources?

Yes, a Virtual Instance has a fixed amount of compute resources. You can change the size of a Virtual Instance through this API.

Is there a limit to the number of additional Virtual Instances I can create in a single Rockset org?

You can have as many additional (Query) VIs as you need. Each Query VI adds a minor CPU overhead to the primary VI (responsible for stream ingest processing) and that overhead is dependent on the write rate of your collections. The lower the write rate on your collections, the lesser the overhead on the primary VI. This means that the size of the primary VI and your write rate determines the maximum number of query VIs. Because Rockset is cloud-native, you can always increase the size of the primary VI if you want to increase the number of query VIs

Does resizing a Virtual Instance incur downtime?

You can resize your Virtual Instance at any time in the Virtual Instances tab of the Rockset Console. There is no downtime associated with this action. Once the resize is completed, Rockset will automatically begin executing queries on your new Virtual Instance.

Can you mount the same collection on multiple Virtual Instances?

Yes, the same collection can be mounted on more than one Virtual Instances. Writes to that collection are processed by the default Virtual Instance. The other Virtual Instances are used to serve queries on that collection.

Can you mount multiple collections on one Virtual Instances?

Yes, you can mount multiple collections on the same Virtual Instance.

If you run a INSERT INTO SELECT query on a Virtual Instance, which compute resoruces

does it use?

The SELECT part of the query uses the compute resources from the Virtual Instance that is specified as part of the query request. This compute is used to generate the result set as specified by the SELECT query. Then the result set has to be inserted into the specified target collection. The compute resources from the default Virtual Instance is used to write the result set into the target collection.