Rockset
  • Querying

Async Queries

Rockset's asynchronous mode enables you to perform the following asynchronously:

  • Send a query request.
  • Receive back a query ID.
  • Poll/fetch results whenever the query completes.

Note: Rockset's maximum query timeout is 30 minutes.

The following subsections describe how asynchronous mode works:

Asynchronous Query Request

Query requests can be sent through either a Query Lambda or raw SQL. To send the request asynchronously, populate the async_options field with the following parameters:

  • client_timeout_ms - Time for which the system will maintain the initial client connection before closing it and returning a partial response signaling that the query is still running. This can be set up to two minutes.
  • timeout_ms - Time after which the query will be terminated by the system and a timeout error returned. This may be set up to 30 minutes.
  • max_initial_results - Maximum number of rows for the client to process in an initial client response. Additional results must fetched through subsequent calls (using a pagination cursor). This parameter does NOT limit the total number of results returned by the query (use the LIMIT SQL clause if you want to do this) – only the cutoff after which results will be paginated. This field only applies if the query completes within client_timeout_ms.

Example Request

The following shows an example of an asynchronous query request configured with async_options:

{
  "sql": {
    "query": "SELECT * FROM _events"
  },
  "async_options": {
    "client_timeout_ms": 10000, # 10 seconds
    "timeout_ms": 108000000,    # 30 minutes
    "max_initial_results": 100
  }
}

Common Asynchronous Flows

Depending on the course of execution within the system, there are several possible results:

Condition(s)Result(s)Comments
Query runs to completion within client_timeout_ms and has a result size smaller than max_initial_results.Returned in the results field in-band.This is identical to how all non-paginated queries return results.
Query runs to completion within client_timeout_ms but with a result size exceeds max_initial_resultsFirst max_initial_results rows returned in-band along with a next_cursor to directly fetch the next page of results via GET .../queries/{queryId}/pages.This is identical to how all paginated queries return results.
Query does not run to completion within client_timeout_ms (regardless of result size).A status: RUNNING response and a query ID.Use this ID to poll the status with GET .../queries/{queryId} until it returns status: COMPLETE. Then you can retrieve the first (and possibly only) page with GET .../queries/{queryId}/pages. This is fairly symmetrical to the paginated query mechanism but involves polling for query completion first.
timeout_ms is exceededThe query will stop execution and return an error.

Note: You can easily force certain behavioral paths. For example, setting max_initial_results or client_timeout_ms to 0 will always force the pagination of results.