Why I Am Joining Rockset

I’m excited to soon be the newest member of Rockset. I will be joining a truly spectacular engineering team, working on a product that leverages deep technical insights to make real-time analytics easy.

My passion is building infrastructure that makes things simpler for users, supporting people at higher levels of the stack by giving them clean APIs and predictable behavior. Network, CPU and disk are all constrained resources, but the most limited commodity in software is space in the developer’s brain. Rockset embodies this philosophy, using sophisticated execution strategies underneath so that you can quickly query the data you already have.

Rockset’s CEO Venkat and I first worked together building TAO, Facebook’s read-optimized graph store. His visionary leadership helped us grow that system to hyper-scale – today TAO handles more than a quadrillion simple queries per day. Venkat’s long-term insights guided us through phase changes in storage, networking and data center technologies, as well as changes in customer needs. He’s doing the same thing at Rockset, thinking about how to fully utilize the cloud of today and tomorrow. I think Snowflake shows that it’s better to rethink data infrastructure for the cloud rather than migrate tightly coupled on-prem systems.

A bit of my code (the F14 hash table) is already used inside Rockset as part of the folly open source library. Tudor, Rockset’s chief architect, authored many of the core parts of folly, and I am incredibly excited to work closely with him. My opinion of him was cemented the first time I contributed a new class to folly. He only took a few minutes to grok the tricky concurrent queue that had taken me weeks to write, then proceeded to extend it into a pipelined processing framework by the next day.

Dhruba, Rockset’s CTO, has also been on my bucket list of people to work with since I met him eleven years ago. He is both strategic and deeply technical, with a track record of identifying and solving the problems nobody has noticed yet. The creation of RocksDB is an obvious example, but my favorite is actually the block placement strategy in HDFS. Dhruba’s improved algorithm dramatically reduced the risk of data loss in large-scale deployments, and he productionized the fix even before it was identified and studied in academia. (If you care about block placement you should read the excellent ‘Copysets’ paper from USENIX ATC’13.)

Rockset is doing what I love, taking responsibility for the hard parts of speed and scalability while making things simple for users. It’s no wonder we’re seeing incredible customer traction, and I’m excited to join the team!