Case Study: Standard Cognition Uses Rockset to Deliver Data APIs and Real-Time Metrics for Vision AI

Walk into a store, grab the items you want, and walk out without having to interact with a cashier or even use a self-checkout system. That’s the no-hassle shopping experience of the future you’ll get at the Standard Store, a demonstration store showcasing the AI-powered checkout pioneered by Standard Cognition.

The company makes use of computer vision to remove the need for checkout lines of any sort in physical retail locations. Their autonomous checkout system only requires easy-to-install overhead cameras, with no other sensors or RFID tags needed on shelves or merchandise. Standard uses the camera information in its computer vision platform to generate locations of individuals in the store—a type of in-store GPS—and track what items they pick up from the shelves. Shoppers simply exit the store with their items and get sent a receipt for their purchases.

Employing computer vision to deliver a no-touch checkout experience requires that Standard efficiently handle large volumes of data from many sources. Aside from video data from each camera-equipped store, Standard deals with other data sets such as transactional data, store inventory data that arrive in different formats from different retailers, and metadata derived from the extensive video captured by their cameras.

As is common with fast-growing markets, Standard’s data and analytics requirements are constantly evolving. Adding external data sources, each with a different schema, can require significant effort building and maintaining ETL pipelines. Testing new functionality on their transactional data store is costly and can impact production. Ad hoc queries to measure the accuracy of the checkout process in real time are not possible with traditional data architectures. To overcome these challenges and support rapid iteration on the product, the Standard engineering team relies on Rockset for their prototyping and internal analytics.

Schemaless Ingest for Running Experiments

Standard builds their production systems to access the streams of events they collect through a number of backend APIs, and the team is continually adding new API endpoints to make more data available to developers. Rockset plays a key role in prototyping APIs that will eventually be productionized and offers several advantages in this regard.

When in the experimental phase, quick schema changes are required when analyzing their data. Rockset does not require schema definition for ingest, but still allows users to run fast SQL queries against the raw data using a very flexible schema-on-read approach. Using Rockset as their prototyping platform, Standard engineers can quickly experiment with different functions on the data.

Standard also uses Rockset for fast prototyping because it can be readily accessed as a fully managed cloud service. Engineers simply connect to various data sources and ingest and query the data without having to manage servers or databases. Compared to the alternative of prototyping on their transactional data store, Standard’s cost of experimentation with Rockset is low.

Ad Hoc Analysis of Operational Metrics

Standard is constantly monitoring operational metrics from retailer partners, and their own demonstration store, to improve the efficiency and precision of their systems. Of particular importance in computer-vision-aided checkout is the accuracy of the transactions. Were shoppers charged for the correct number of items? How accurate were the AI models compared to human-resolved events?

The engineering team pulls together multiple data sets—event streams from the stores, data from vendors, store inventory information, and debug logs—to generate accuracy metrics. They stream all this data into Rockset, which allows Standard to run ad hoc queries to join across data sets and analyze metrics in real time, rather than wait for asynchronous data lake jobs.

An Environment for Rapid Prototyping and Real-Time Analytics

Standard incorporates Rockset into their development flow for rapid prototyping and real-time analytics purposes. They bring in transactional data and various third-party data sets, typically in CSV or Parquet format and each with its own custom schema, using the Rockset Write API for ingestion whenever new data is available.

For feature prototyping, engineers build an experimental API, using the Rockset Node.js client, that is refined over multiple iterations. Once a feature is mature, it is converted to a serverless function, using Google Cloud Functions, in their online production system in order to present data as an API to developers. This flow allows the engineering team to move quickly, with no infrastructure required, when developing new functionality. Standard productionizes several endpoints a day using this methodology.

In the real-time analytics scenario, data from disparate sources—structured data managed by Standard and unstructured third-party data—is loaded into Rockset. Once ingested into Rockset, engineers can immediately perform SQL queries to measure and analyze operational metrics. Rockset offers the Standard team an ideal environment for ad hoc queries, allowing engineers to bring in and query internal and external data sets in real time without having to worry about indexing the data for performance.

Constantly Improving Checkout Accuracy and Product at Standard

Standard’s Rockset environment allows the team greater speed and simplicity when developing new features and verifying the accuracy of their AI models. In a nascent market where correctness of the computer vision platform will be crucial in gaining adoption of its automated checkout system, the ability to constantly improve accuracy and product functionality gives Standard an important edge.

“The team at Standard is always looking to increase the accuracy of the computer vision platform and add new features to the product. We need to be able to drive product improvements from conception to production rapidly, and that involves being able to run experiments and analyze real-time metrics quickly and simply,” says Tushar Dadlani, computer vision engineering manager at Standard Cognition. “Using Rockset in our development environment gives us the ability to perform ad hoc analysis without a significant investment in infrastructure and performance tuning. We have over two thirds of our technical team using Rockset for their work, helping us increase the speed and agility with which we operate.”

As Standard continues to evolve its AI-powered autonomous checkout offering, the team hopes to bring even more data into its platform in the future. Standard will extend the same rapid development model, enabled by Rockset, to incorporating new types of data into its analysis. Its next project will introduce user behavior event streams into its analysis, using Rockset’s SQL engine to join across the multiple data sets being analyzed.