Scaling MongoDB: Best Practices for Sharding, Indexing and Performance Isolation
A tech talk on how organizations tackle the challenges of sharding, indexing and offloading queries to scale MongoDB
Developers and data teams typically make early decisions on data modeling, shard configuration and indexing strategy in MongoDB without full knowledge of future query access patterns and database loads. These decisions have the potential to affect how MongoDB scales to handle data growth and new application requirements, so how should engineering organizations tackle the challenges of scaling MongoDB?
Dai Shi, a former Site Reliability Engineer at Foursquare, spent several years operating MongoDB in production at scale, writing a custom cluster balancer and building automated self-healing tools to ensure cluster health. The lessons learned from his experience culminate in 3 scaling strategies that he’ll cover in this talk:
Sharding & cluster balancing: How to choose your shard key and how to make the cluster autobalancer work for you
Indexing: Processes for creating new collections and indexes, and how to identify when a production issue is caused by an index
Offloading from production MongoDB: Identifying workloads to offload to ensure reliability and isolation of your primary MongoDB database, and methods for building integrations to secondary data stores
The talk will conclude with a live Q&A so you can get your scaling questions answered by Dai.
About the Speakers
Dai Shi has been a Site Reliability Engineer at Rockset since 2017, helping to build out all parts of Rockset’s infrastructure. He was also part of the production team at Foursquare from 2013 to 2017, where he spent a lot of time working on scaling and building tools for MongoDB, led a cross functional team focusing on MongoDB related projects, and participated in MongoDB’s Masters program.
Prakash Chockalingam is the Director of Product Management at Rockset and has held product management and engineering roles at Databricks, Netflix and Yahoo.