Case Study: Zembula and Rockset Power Real-Time Marketing Email Personalization
June 3, 2022
Zembula is a Portland, Oregon-based venture-backed startup that is breaking new ground in real-time customer personalization.
Expanding Smart Banners to all kinds of promotional emails caused our traffic to explode 10x. We needed a lower-ops, cost-effective and scalable database to pave the way for our next 100x of growth. — Robert Haydock, CEO, Zembula
We have developed technology enabling companies to deliver emails that are dynamic and hyper relevant to every recipient. The moment an email is opened, a Smart Banner™ embedded in the email pings our campaign decision engine. Using up-to-the-moment customer data, our analytics engine instantly updates the image-rich banner with an offer or useful information tailored to that customer. These include price drops on items sitting in an abandoned shopping cart, shipment tracking locations, loyalty points or birthday rewards.
Smart Banners™ boost the performance of 100 percent of your customer emails. They can display items in a customer’s abandoned shopping cart and other transactional info in a promotional email, or they can remind you what free offers your loyalty points qualify you for in an abandoned cart email. By giving companies up to 70 different image and message combinations to choose from, Smart Banners™ also minimizes the repetitiveness that can create customer fatigue.
Some of the most innovative, fast-rising online retailers use Zembula, including Purple (mattresses), Thrive Causemetics, and Manscaped. They are seeing a 10-17 percent revenue increase in their promotional emails. Considering the volume of emails sent by B2C companies – sometimes twice a day – that can lead to tens of millions of dollars in additional sales, or more.
We are also helping travel and hospitality companies such as Vail Resorts drive bookings and promote specials. And we also help financial service companies better connect with customers.
No wonder our customers tell us that Smart Banners™ “feels like magic” to them. However, the magic almost didn’t happen, because of scalability and cost issues in our now-legacy infrastructure.
Challenge Unmet: Scaling Our Legacy Analytics Engine
Zembula CTO Carl Thorner and I founded Zembula in 2013. With a small but nimble technical team, we deliberately chose cloud-native and managed web services for our backend architecture.
Because our usage is so spiky and can overwhelm the concurrency limits of their APIs, we cache lookup and API data from third-party services in Amazon DynamoDB for speed and reliability.
Ironically, the main exception to our low-ops serverless agenda was the centerpiece of our infrastructure: our analytics engine. We had been using Elasticsearch since our founding, going back to Elasticsearch 1.5.
Elasticsearch underpinned two key processes:
- The SmartBanners updated instantly when consumers opened their emails
- The internal dashboard used by our corporate customers to oversee the success of their email campaigns. This ingests and analyzes data gathered by our systems such as open rates, clicks on Smart Banners, generated sales, etc.
As we grew, Carl had been able to maintain sub-second query times in Elasticsearch through a combination of more servers and heavy engineering work.
However, this came at a high cost. By the beginning of 2022, we were maintaining numerous Elasticsearch servers hosted in Amazon EC2, for which we had to pay upfront Elasticsearch server licenses as well as ongoing EC2 compute and storage costs. This was also wasteful. Our usage was and is extremely spiky. Sending out one customer email blast could result in several million opens in a few seconds. In between those peaks were long valleys, when our expensively deployed servers were underutilized. We tried to optimize Elasticsearch to store data more efficiently, but this was a huge drain on Carl’s time.
Things came to a head when we expanded Smart Banners™ from a limited set of emails to all kinds of promotional emails. Our traffic exploded, growing 10x in just three months.
We could have kept adding more Elasticsearch servers, but it simply was not a cost-effective or efficient path forward. We needed to rearchitect for the next 100x of growth. — Robert Haydock, CEO, Zembula
Finding a Lower-Ops, Highly Scalable, Price Optimized Alternative
The first alternative we investigated was ChaosSearch, which is a drop-in replacement for Elasticsearch. Unfortunately, this didn’t prove to be so smooth for us, as we were not running the necessary version of Elasticsearch. And while ChaosSearch was extremely cost-effective, we could not achieve the query speeds and results we needed, even after a lot of engineering work by us and the helpful ChaosSearch team.
Our CTO Carl actually considered building a whole new analytics stack using open-source tools. Fortunately, before we got to that state, he found Rockset in February 2022, and immediately signed up to trial its real-time analytics platform.
Rockset’s performance was super fast, and the support from the Rockset team was great. We also loved Rockset’s autoscaling, which optimized price-performance and was one of many low ops features of this fully managed platform that we loved.
We also trusted Rockset’s technical foundation, based on the RocksDB database. As a key-value store, RocksDB is both extremely fast and scalable. It supported Facebook’s analytics during its hypergrowth years. At the same time, Rockset has engineered RocksDB so that it fully supports complex SQL queries, which was lacking in Elasticsearch.
That meant we could deploy Rockset not just for instant email personalization, but also to power our big data dashboard for customers to track the outcomes of their email campaigns. Because Rockset has a high standard for security, including SOC 2 certification, we feel confident that data privacy will be maintained in line with our SOC 2 requirements.
Time Flies – as Does Rockset
Within a week, we had decided to switch our analytics engine from Elasticsearch to Rockset. Enabled by Rockset’s DynamoDB connector, we went live with Rockset just two weeks later.
Our Smart Banners™ are faster with Rockset, with 90 percent of them delivered in 100 milliseconds or less. The rest are refreshed to consumers in half a second or less. And Rockset’s fully managed platform freed Carl and his team from time-consuming operational work. All told, when including its reduced management burden, Rockset is 3x more cost-effective for us than Elasticsearch.
Our customer analytical dashboard also greatly improved with Rockset. Using Amazon Kinesis Firehose in combination with Rockset’s rollups feature, we could ingest huge amounts of data efficiently and quickly from S3 into Rockset. Though we could have made the data ready for queries in a few minutes, we deliberately chose a 15-minute SLA to avoid the cost of hitting S3 too frequently.
Agile scalability and cost-effectiveness were the two improvements we sought the most for our analytics. With Rockset, we gained both of those and more. For one, we are excited about Rockset’s expansion of its support for machine learning. Our campaign decision engine already relies on ML-trained algorithms, but they were built using a limited dataset due to the high cost of integrating disparate datasets. That’s not an issue with Rockset, which naturally aggregates data as it is ingested.
We are looking forward to using Rockset to improve the recommendations in not just our customer emails, but also our recently-launched Smart Banners for SMS and text messages, as well as push and in-app notifications on your smartphone or tablet. Rockset will likely soon become the main hub for our most-advanced, real-time analytics.
Rockset has been an unqualified success story for us. After maintaining Elasticsearch as a backup for two months, we turned it off for good – and haven’t looked back.
Rockset is the leading real-time analytics platform built for the cloud, delivering fast analytics on real-time data with surprising efficiency. Learn more at rockset.com.