Real-time analytics trend sweeping the data industry - Venkat Venkataramani and Dhruba Borthakur, Rockset | CUBE Conversation

Dhruba Borthakur:

Lisa Martin:

Excited to learn more about Rockset. Venkat, talk to me about Rockset and how it's putting

real-time-analyticswithin the reach of every company.

Venkat Venkataramani:

If you see the Confluent IPO, if you see where the world is going in terms of analytics, we look at this real-time analytics is like the lost frontier. Everybody wants fast queries on fresh data, nobody wants to say, "I don't need that. Give me slow queries on stale data." I think if you see what data warehouses and data lakes have done, especially in the cloud there, they've really, really made batch analytics extremely accessible, but real-time analytics still seems too clumsy, too complex, and too expensive for most people, and we are on a mission to make real-time analytics, make it very, very easy and affordable for everybody to be able to take advantage of that. So that's what we do

Lisa Martin:

And you're right. Nobody wants stale data or slower queries. And it seems like one of the things that we learned, Venkat sticking with you, in the last 18 months of the very strange world that we're living in is that real-time is no longer a nice to have, it's really a differentiator and table stakes for businesses in every industry. How do you make it more affordable and accessible to businesses in so many different industries?

Venkat Venkataramani:

I think that's a great question. I think there are, at a very high level, there are two categories of use cases we see. I think there is one full category of use cases where business teams and business units are demanding business observability. If you think about one domain that actually understood real-time and made everything work in real-time is the DevOps world, metrics and monitoring coming out of all these machines and because they really want to know as soon as something goes wrong, immediately I want to be able to dive in and click and see what happens, but now businesses are demanding the same thing, right? Like a CEO wants to know, "Are we on track to hit our quarterly estimates or not? And tell me now, what's happening?" Because the larger the company, the more complex the revenue operations dashboards are and if you don't give them real-time visibility, the window of opportunity to do something about it disappears, and businesses are really demanding that. And so that is one big use case we have.

Venkat Venkataramani:

And the other strange thing we're also seeing is that customers are demanding real-time, even from the products they're using. So you could be using a SaaS product for sales automation, support automation, marketing automation. Now I don't want to use a product if it doesn't have real-time analytics baked into that product itself. And so all these software companies providing a SaaS service to their customers and clients, they are also looking to actually, their proof of value really comes from the analytics that they can show within the product and if that is not interactive in real-time, then they're also going to be left behind. So it's really a huge differentiator, whether you're building a software product or you're running a business, the real-time observability gives you a window of opportunity to actually do something about, when something goes wrong, you can actually act on it very, very quickly.

Lisa Martin:

Right, which is absolutely critical. Dhruba, I want to get your take on this as the CTO and co-founder, as I introduced you, what were some of the gaps in the market back in 2016 that you saw that really necessitated the development of this technology?

Dhruba Borthakur:

Yeah. For real-time analytics, the difference compared to what it was earlier is that earlier things used to be a lot of batch processes. Again, the reason being because there was something called map reduce and that was a scanning system, that was a kind of an invention from Google. We talked about processing big data sets, and it was about scanning, scanning large data sets to give answers whereas for real-time analytics, what the new trend is that how can you index these big data sets so that you can answer queries really fast? So this is what Rockset does as well, is that we have capabilities to index humongous amounts of data cheaply, efficiently, and economically feasible for our customers, and that's why query is there to leverage the index to give fast queries. This is one of the big changes.

Dhruba Borthakur:

The other change obviously is that it has moved to the cloud, right? A lot of analytics have moved to the cloud. So Rockset is built natively for the cloud, which is why we can scale up, scale down resources when queries come and we can provide great SLAs for people as far as data latency and as far as query latency is concerned, both of these things. So these two trends, I think, are the power behind moving, making people use more real-time analytics,

Lisa Martin:

Right and as Venkat was talking about how it's an absolute differentiator for businesses, last year we saw all these quick pivots to survive and ultimately thrive and we're seeing the businesses now coming out of this that were able to do that and we're able to pivot to digital, to be successful, and to out-compete those who maybe were not as fast. I saw that recently Venkat, you guys had a new product release a few weeks ago, major product release, that is making real-time analytics on streaming data sources like Apache Kafka, Amazon Kinesis, Amazon DynamoDB, and data lakes a lot more accessible and affordable. Break down that launch for me and how is it doing the accessibility and the affordability that you talked about before?

Venkat Venkataramani:

Extremely good question. So we are really excited about what we call SQL based rollups is what we call that release. So what does that do? So if you think about real-time analytics and even teeing off the previous question you asked on what a gap in the market, the gap in the market is really all warehouses and lakes are built for batch. They're really good at letting people accumulate huge volumes of data and once a week, analyst asking a question, generating a report, and everybody's looking at it. And with real-time, the data never stops coming, the queries never stop coming, so how do you... If I on real-time metrics on all these huge volumes of data coming in, now, if I drain it into a huge data lake and then I'm doing analytics on that, it gets very expensive and very complex very quickly.

Venkat Venkataramani:

And so the new release that we had is called SQL based rollups, where simply using SQL, you can define any real-time metric that you want to track across any dimensions you care about, it could be geo, demographic, and other dimensions you care about that, and Rockset will automatically maintain all those real-time metrics for you in real time in a highly accurate fashion, so you'll never have to doubt whether the metrics are valid and it'll be accurate up to the second. And the best part is you don't have to learn a new language, you can actually use SQL to define those metrics and Rockset will automatically maintain that and scale that for you in the cloud and that I think reduces the barrier. So if somebody wants to build a real-time, track something for their business in real-time, you have to duct tape together multiple disparate components and systems that were never meant to work with each other. Now you have a real-time database built for the cloud that supports full feature SQL. So you can do this in a matter of minutes, which would probably take you days or weeks with alternate technologies.

Lisa Martin:

That's a dramatic X reduction in time there. I want to mention the snowflake IPO since you guys mentioned the confluent IPO, you say that Rockset does for real-time, what snowflake did for batch. Dhruba, I want to get your perspective on that. Tell me about that. What do you mean by that?

Dhruba Borthakur:

Yeah. So we see this trend in the market where a lot of analytics, which are very batch, they get a lot of value if they move more real-time, right? Like Venkat mentioned, when analytics powers actual products which need to use analytics into their, to make the product better. So Rockset very much plays in this area. So Rockset is the only solution, I shouldn't say it's the solution, it's a database, it's a real-time database, which powers these kind of analytic systems. If you don't use Rockset, then you might be using maybe a warehouse or something, but you cannot get real-time because there is always a latency of putting data into the data warehouse. It could be minutes, it could be hours. And then also, you don't get too many people making concurrent queries on the warehouse.

Dhruba Borthakur:

So this is another difference for real-time analytics because it powers applications, the query volume could be large. So that's why you need a real-time database and not a real time warehouse or any other technologies for this. And this trend has really caught up because most people have either are pretty much into this journey, you answered me this previous question about what has changed since 2016 as well, and this is a journey that most enterprises we see are already embarking upon.

Lisa Martin:

One thing too that we're seeing is that more and more applications are becoming data intensive applications, right? We think of whether it's Instagram or DoorDash or whatnot, or even our banking app, we expect to have the information updated immediately. How do you help, Dhruba sticking with you, how do you help businesses build and power those data intensive applications that the consumers are demanding?

Dhruba Borthakur:

That's a great question. And we had both me and Venkat, we had seen these data applications at large scale when we were at Facebook earlier, we were both parts of the Facebook team. So we saw how real-time was really important for building that kind of a business, and that was social media but now we are taking the same kind of backends, which can scale to huge volumes of data to the enterprises as well. Venkat, do you have anything to add?

Venkat Venkataramani:

Yeah, I think when you're trying to go from batch to real-time, you are 100% spot on that a static report, a static data dashboard actually becomes an application, becomes a data application. And it has to be interactive. So you're not just showing a newspaper where you just get to read, you want to click and deep dive, do slice and dice the data, to not only understand what happened but why it happened and come up with hypothesis to figure out what I want to do with it, so the interactivity goes is important and the real-timeliness, it becomes important. So the way we think about it is once you go into real-time analytics, the data never stops coming, that's obvious, data freshness is important, but the queries never stop coming also because when your dashboards and metrics are getting up to date real-time, you really want alerts and anomaly detection to be automatically built-in, and so you don't even have to look at the graphs once a week.

Venkat Venkataramani:

When something is off, the system will come and tap on your shoulder and say, "Hey, something is going on." And so that really is a real-time application at that point, because it's constantly looking at the data and querying on your behalf and only alerting you when something actually is interesting happening that you might need to look at. So yeah, the whole movement towards data applications and data intensive apps is a huge use case for us. I think most of our customers, I would say, are building a data application in one shape or form or another.

Lisa Martin:

And if I think of use cases like customer 360, as customers and consumers of whatever product or solution we're talking about, we expect that these brands know who we are, know what we've done with them, what we've bought, what to show me next is what I expect, whether again, it's my bank or it's Instagram or something else. So that personalization approach is absolutely critical and I imagine another big game changer differentiator for the customers that use Rockset, what do you guys think about that?

Venkat Venkataramani:

Absolutely. Personalized recommendation is a huge use case. We see this all over. We have, Ritual is one of the customers, we have a case study on that, I think, they want to personalize, they generate offline recommendations for anything that the user is buying, but they want to use behavioral data from the product to personalize their experience and combine the two before they serve anything on the checkout lane, right? We also see in B2B companies, real-time analytics and data applications becoming very important and thing and we have another customer Command Alkon, they have a supply chain platform for heavy construction and 80% of concrete in North America flows through their platform for example. And what they want to know in real-time is reporting on how many concrete trucks are arriving at a big construction site, which ones are late and whatnot.

Venkat Venkataramani:

And the real-time analytics needs to be accurate and needs to be up to the second, "Don't tell me what trucks were coming an hour ago, no, I need this right now." And so even in a B2B platform, we see this very similar trend where real-time reporting, real-time search, real-time indexing is actually a very, very important piece to the puzzle, and not just for B2C examples that you said. And the Instagram comment is also very appropriate because one of a hedge fund customer came to us and said, "I have dashboards built on top of snowflake. They're taking two to five seconds and I certain parts of my dashboards where I am actually having 50, 60 visualizations. You do the math, it takes many minutes to load." And so they said, "Hey, you have some indexing tech, can you make this faster?" Three weeks later, the queries that would take two to five seconds on a traditional warehouse or a cloud data warehouse, came back in 18 milliseconds with Rockset.

Venkat Venkataramani:

And so it is so fast that they said, "If my internal dashboards are not as fast as Instagram, no one in my company uses it." These are their words. And so they're really, the speed is really, really important, the scale is really, really important, data freshness is important. If you combine all these things and also make it simple for people to access with SQL based, that's really the real unique value prop that we have with Rockset which is what our customers love.

Lisa Martin:

You brought up something interesting, Venkat, that kind of made me think of the employee experience. We always think of the customer 360, the customer experience, and the employee experience, in my opinion, is inextricably linked. The employees have to have access to what they need to deliver and help these great customer relationships and as you were saying, the employees are expecting databases to be as fast as they see on Instagram when they're surfing on their free time, then adoption, I imagine, gets better and obviously, than the benefit from the end user and customer's perspective is that speed. Talk to me a little bit about how Rockset, and I would like to get both of your opinions here, is the facilitator of that employee productivity for your customers?

Venkat Venkataramani:

This is a great question. In fact, the same hedge fund customer, I pushed them to go and measure how many times do people even look at all the data that you produce, how many analysts and investors actually use your dashboards and asked them to go investigate that. And one of the things that they eventually showed me was there was a huge uptake when their dashboards went from two to three second lags to 18 milliseconds. They almost got the daily active user for their own internal dashboards to be almost going from five people to the entire company. So I think its spot on. So it really goes back to really leveraging the data and actually doing something about it. If I ask a question and system is going to take 20 minutes to answer that, I will probably not ask as many questions as I want to.

Venkat Venkataramani:

When it becomes interactive and very, very fast, and all of a sudden, I not only start with a question and I can ask a follow-up question and then another follow-up question and make it really drive that to a conclusion I can actually act upon it, and this really accelerates this. So even if you look at the macro, you hear these phrases, "The world is going from batch to real-time." And in my opinion, when I look at this, people want to accelerate their growth. People want to make faster decisions. People want to get to, "What can I do about this?" And get action insights. And that is not really is going to come from systems that take 20 minutes to give a response, it's going to really come from systems that are interactive and real-time. And that's really, the need for acceleration is what's really driving this movement from batch to real-time and we're very happy to facilitate that and accelerate that movement.

Lisa Martin:

And it really drives the opportunity for your customers to monetize more and more data so that they can actually act on it, as you said, in real-time and do something about it, whether it's a positive experience or it is remediating a challenge. Last question guys, since we're almost out of time Dhruba, I want to understand from you, talk to me about the Rockset AWS partnership and what the value is for your customers.

Dhruba Borthakur:

Okay. Yeah. I'll get to that in a second, but I wanted to add something to your previous question. I think my opinion for all the customers that you see is that real-time analytics is addictive. Once they get used to it, they can't go back to the old stuff. So this is what we have found with all are customers. So yeah, for the AWS question, I think maybe think Venkat can answer that better than me.

Venkat Venkataramani:

Yeah. We love partnering with AWS. I think they're the world leader when it comes to public clouds, we have a lot of joint happy customers that are all AWS customers. Rockset is entirely built on top of AWS and we love that. And there is a lot of integrations that Rockset natively comes with, so if you are already managing your data in AWS, there are no data transfer costs or anything like that involved for you to also index that data in Rockset and actually build real-time applications and stream the data to Rockset. So I think the partnership goes in very, very deep in terms of like, we are an AWS customer, we are a partner and our go to market teams work with them. And so, yeah, we are very, very happy, AWS fanboys here. Yeah.

Lisa Martin:

Excellent. It sounds like a very great synergistic collaborative relationship, and I love, Dhruba, what you said. This is a great quote, "Real-time analytics is addictive." That sounds to me like a good addiction for businesses and every industry to take up. Guys, it's been a pleasure talking to you. Thank you for joining me, talking to the audience about Rockset, what differentiates you, and how you're helping customers really improve their customer productivity, their employee productivity, and beyond. We appreciate your time.

Venkat Venkataramani:

Thanks, Lisa.

Dhruba Borthakur:

Dhruba Borthakur:

Lisa Martin:

For my guests. I'm Lisa Martin. You're watching this CUBE Conversation.