Async Queries

Rockset's asynchronous mode enables you to perform the following asynchronously:

• Send a query request.
• Receive back a query ID.
• Poll/fetch results whenever the query completes.

Note: Rockset's maximum query timeout is 30 minutes.

The following subsections describe how asynchronous mode works:

• Asynchronous Query Request
• Common Asynchronous Flows

Asynchronous Query Request

Query requests can be sent through either a Query Lambda or raw SQL. To send the request asynchronously, populate the async_options field with the following parameters:

• client_timeout_ms - Time for which the system will maintain the initial client connection before closing it and returning a partial response signaling that the query is still running. This can be set up to two minutes.
• timeout_ms - Time after which the query will be terminated by the system and a timeout error returned. This may be set up to 30 minutes.
• max_initial_results - Maximum number of rows for the client to process in an initial client response. Additional results must fetched through subsequent calls (using a pagination cursor). This parameter does NOT limit the total number of results returned by the query (use the LIMIT SQL clause if you want to do this) – only the cutoff after which results will be paginated. This field only applies if the query completes within client_timeout_ms.

Example Request

The following shows an example of an asynchronous query request configured with async_options:

{
  "sql": {
    "query": "SELECT * FROM _events"
  },
  "async_options": {
    "client_timeout_ms": 10000, # 10 seconds
    "timeout_ms": 1800000,    # 30 minutes
    "max_initial_results": 100
  }
}

Note: You can use async_options in query lambdas in exactly the same way

Common Asynchronous Flows

Depending on the course of execution within the system, there are several possible results:

Note: You can easily force certain behavioral paths. For example, setting max_initial_results or client_timeout_ms to 0 will always force the pagination of results.