Scaling Real-Time Gaming Leaderboards with DynamoDB and Rockset
July 22, 2021
Social gaming is on the rise. During COVID-19, 29% of consumers reported playing games on a weekly basis and the goal for many players was to connect with friends and family (Deloitte: Games and Streaming Services Fight it Out During Pandemic from VentureBeat). One of the challenges that gaming companies face is rapidly building features that can strengthen network effects. Investments in these features will keep players coming back, enabling effective monetization of games.
One of those social features is a gaming leaderboard. A leaderboard provides a scored ranking of the players in the game to encourage healthy competition. Gamers can see their position relative to others in the game, motivating them to increase their in-game activity to compete and advance.
While the concept of the leaderboard is not new, many gaming companies have taken leaderboards to the next level. They provide social sharing and commenting options to allow users to brag about their results in gaming forums. And, new flavors of leaderboards have popped up to allow for gamers to compete with small groups of friends, geographic areas, celebrity gamers and more. These new characteristics of leaderboards are enhancing the social aspects of gaming.
A leaderboard is one example of real-time analytics in gaming. Gamer data has to be constantly updated and reflected in real-time in the leaderboard. And, a leaderboard requires complex analytics as multiple attributes need to be taken into consideration to rank gamers. Meeting these real-time analytics requirements at the scale of thousands to millions of gamers is a challenge.
In this blog, we’ll discuss the speed and scale requirements of gaming leaderboards. We’ll then share how Rockset is able to meet these challenging requirements and devise a new data stack for real-time analytics in gaming using Amazon DynamoDB and Rockset.
Requirements for Leaderboards
Leaderboards are computationally intensive, requiring multiple aspects of gameplay to be taken into consideration when calculating a ranking.
Building these complex analytical features are also doubly challenging in the gaming industry given the number of gamers and the size of data. A popular game can easily attract millions of gamers worldwide and reach 1-5 TBs of data a day.
Leaderboards and other real-time analytics features in gaming have a unique set of requirements. They need to:
- Scale to millions of gamers and thousands of concurrent gamers
- Reflect real-time gamer activity for an accurate view of gamer rankings
- Support low-latency aggregations and joins to rank gamers based on multiple attributes
- Incorporate new signals as the game evolves and criteria for rankings shift
- Ensure high availability
Many systems have not been designed to meet the challenging requirements of leaderboards. They either lack the speed- the query speed or support for real-time data- or cannot achieve the scale required of applications.
Why Rockset for Leaderboards
Rockset is purpose-built to serve real-time analytics at scale. Rockset uses an indexing approach to deliver millisecond latency complex analytics to millions of gamers.
Indexes are an additional data structure that works like a lookup table on the indexed fields, enabling fast data retrieval at query time. FAANG companies like Google and Facebook have employed indexing systems to power text search, newsfeed content, spam detection, advertising and more.
Rockset took the same approach of the FAANGs- index everything. That means, any semi-structured, geo or time series data is ingested and indexed in multiple ways for fast queries out-of-the-box. This removes the need for tedious, time-consuming index creation and management. Rockset's Converged Index™ efficiently enables queries on new data to be available almost instantly and to perform incredibly fast. It also enables developers to easily incorporate new signals as the game or the criteria evolve. We think the flexibility of indexing everything is the key to releasing new social gaming features rapidly.
Rockset meets the challenging requirements for leaderboards with the following core features:
- One-second data latency: Rockset processes data in real-time and makes new data queryable in less than a second. Rockset uses RocksDB's LSM trees to cache incoming writes and makes writes visible to existing queries as soon as they happen.
- Fully mutable Converged Index™: Mutability is key to processing constantly changing data such as gamer profile, behaviors and more and making that available for querying.
- SQL on schemaless data: Rockset delivers search, aggregations and joins on any data without requiring any data modeling or performance tuning. This makes it easy for gaming companies to analyze data in modern formats.
- Severless service: Rockset is a distributed database that can be scaled with a click of the button or an API call. This makes it easy for gaming companies to access fast analytics while staying lean.
DynamoDB + Rockset: The Ultimate Gaming Analytics Stack
We’ve devised the ultimate gaming analytics stack by pairing Amazon DynamoDB, a leading NoSQL key-value database, with Rockset. This stack enables game developers to get the best of both worlds: fast writes and heavy reads.
DynamoDB is used by leading gaming companies to process 100s of thousands of requests per second. Gaming companies including Electronic Arts (EA), FanDuel, Riot Games, Gametime and Pennypop all use DynamoDB to store game player data. That’s because DynamoDB is a well-known serverless service that excels at fast writes at extreme scale.
While DynamoDB is an excellent system for write-heavy workloads, it was not designed to support complex analytics. That’s why it’s a best practice to pair DynamoDB with an analytics system like Rockset. As Rockset indexes DynamoDB data, queries and data can easily be added or modified without impacting the data modeling in DynamoDB. This gives developers greater flexibility to build new social features and functionality.
Leaderboards data stack using DynamoDB and Rockset.
Rockset has a built-in connector to DynamoDB streams to continuously stay in sync with the latest data. As soon as new data arrives into Rockset it is indexed for serving millisecond-latency search, aggregations and joins. The full data stack for analytics- DynamoDB and Rockset- is serverless, freeing teams from managing infra ops.
eGoGames, an esports platform for mobile gaming, uses DynamoDB to store all the master data related to players and matches on their platform. They use Rockset to query DynamoDB data within seconds of updates to optimize the gamer experience. eGoGames selected Rockset for real-time analytics because it allowed them to get up and running without spending time on coding or managing systems. And, Rockset could be used for a wide range of gaming analytics use cases.
eGoGames is a leading eSports platform that uses Rockset for real-time analytics.
The Future of Social Gaming: Real-Time Analytics
Leaderboards are just one example of a growing wave of social features in gaming. Gaming companies are using real-time analytics to improve the customer experience with matchmaking, increase engagement with activity feeds, optimize ad experiences and more.
These social features have a set of requirements around query latency, complex analytics, data latency and scale that are challenging to achieve using existing data stacks. That’s why Rockset is introducing a new data stack for gaming analytics that enables developers to release new social features rapidly.
See for yourself how easy it is to serve leaderboards and other social features using Rockset with a 14-day free trial and $300 in credits.