Case Study: iYOTAH Brings Real-Time IoT Analytics to Dairy Farming with Its AgTech SaaS Platform

The American dairy industry is a mighty one. America’s 32,000 dairy farmers not only produce the most milk in the world, they are also the most efficient, producing 23 thousand pounds of milk per cow per year — almost 20 times the weight of an average (1,200 pound) dairy cow.

For their genetically strong herds, healthy cows, high yields, even increasingly green operations, farmers can credit both agricultural science as well as data science. American dairy farmers were early adopters of using data to improve their operations, to track the genetic markers of their livestock, to monitor forecasts for weather and feed prices, installing IoT sensors to track the cow’s movements, and recording actual milk production numbers.

But as in most industries, few farmers have kept up with the latest advances in data analytics, especially in the real-time and streaming arena, hurting efficiencies and profits. “To develop the [dairy] industry further,” mused leading dairy industry research group, IFCN, in late 2021, “better connectivity and digitalization” are needed.

That's what iYOTAH Solutions aims to deliver. In August of 2019, the Colorado-based company launched and began development of a real-time SaaS analytics platform to bring digital transformation to American dairy farmers.

Grabbing Data By the Horns

What determines how much milk a cow will produce? Its basic DNA for one, but also how its genes actually translate into physical characteristics, or its phenotype. The environment it lives in is key -- how well-fed it is, if it gets cold or sick, how much exercise and activity it gets, etc.

Farmers tracked that data by hand when dairy farms were small enough for them to be on a first-name basis with their cows. No longer. The average farm keeps 234 cows today, but the majority of the milk comes from herds that are anywhere from 5000-100,000. To manage them effectively, farmers have long used PC-based applications to track key data. More recently, farmers have started automating the process of tracking and data entry by using “Fitbits for cows” and other IoT sensors to track their cows’ movement, fertility, feed intake, milk production, and even their behavior.

“One of the many things I learned when I got into this industry was that it’s true: happy cows do make more milk,” said Pedro Meza, VP of engineering at iYOTAH.

However, as farms continue to grow and profit margins continue to thin, dairy farmers are looking for more efficient and powerful ways to use their data. But they’ve been stymied. Most continue to use older Windows software that track specific areas, such as herd info and breeding history, feed, or milk production, including samples of fat and protein content that determine the milk’s market value. “Other data, such as finances, are tracked in Excel or Quickbooks,” said Meza, or even remain stuffed as “receipts in the shoebox.”

“Dairy farms are multimillion dollar operations, yet farmers tell us that 30 percent of their time is spent on gathering their data,” Meza said.

When data is siloed and non-digitized, it can’t be analyzed for historical trends, nor can it be combined to make smarter decisions. For instance, joining two data tables showing hourly temperatures and humidity and how much feed the cows have consumed could allow farmers to improve feeding efficiencies and optimize milk production.

Tipping Point

iYOTAH set out to build what today’s farmers need: a modern, unified solution platform that gives them a high-level view of their operations, real-time alerts with controllable thresholds, and drill-down interactivity for combining and exploring data with minimal latency.

Rather than forcing farmers to quickly abandon their tried-and-trusted applications, iYOTAH decided to create a set of software agents that install themselves on the farmers’ PCs. Every predetermined time period, the agents would scan the applications for newly-entered or uploaded data -- everything from highly-compressed herd genetic data, to dimensional models. When a change is detected, the data is ingested into a data lake hosted on Amazon S3. There, the data is converted, tagged with metadata, cleaned, and de-duplicated in preparation for queries.

For a high-performance database that could quickly serve the queries to their dashboards, iYOTAH looked at several options. They demoed but quickly eliminated Snowflake. They also looked at using AWS-hosted Spark as its database engine and serving up queries to a Tableau dashboard. Meza and his team also voted against this approach, saying it locked them into an expensive infrastructure that “didn’t quite meet their long-term needs.”

In the end, iYOTAH decided to build its application from scratch and use Rockset as the real-time query engine. Though this would entail greater investment in building out their dashboards, iYOTAH “wanted to be in control of our own roadmap,” said Meza. And Rockset made the process of building the data application and pipelines much faster. With Rockset’s built-in connector to S3, enabling automatic exports from S3 to Rockset was easy. Data is uploaded to Rockset from S3 every 3-5 minutes.

Rockset also powerfully supports SQL, with which all of Meza’s developers were experts. Rockset also boasts time-saving features such as Query Lambdas -- named, parameterized SQL queries stored on the Rockset database that can be executed from a dedicated REST endpoint. This makes queries easier for developers to manage and optimize, especially for production applications.

All of this data feeds a single application divided currently into ten dashboards that can be customized displaying a total of 150 different visualizations with all of the data served up by Rockset. One dashboard displays near-real-time sample data of its milk’s nutritional content (fat and protein levels), which determines the milk’s market value. Another focuses on breeding, tracking the cows through pregnancy and beyond, notifying farmers when it is time to breed them and then using genetic data to match them with the right sires for more milk production.

Rockset also powers real-time monitoring of animal health, and tracking feed and manure levels. The farmers can configure alerts so that they are notified if the temperatures rise or drop below a certain mark -- key as cold or extreme heat for cows cause less milk production and can cause an increase in illness. Data from each of these charts can be correlated or overlayed with other charts. Farmers can also drill down into their charts in real time to explore and get questions answered interactively.

Moving Forward

Using the iYOTAH platform, one of their test farms was able to integrate all of its operational data for the first time in order to analyze and optimize its feed efficiency. That helped the farm reap $781,000 in increased revenue from better-fed cows that produced more milk and savings from less wasted feed, for which the iYOTAH team were recognized (above) as the winner of an Indiana state AgriBusiness Innovation Challenge.

This real-time dashboard for farmers is only the beginning. iYOTAH is working with the National Dairy Herd Information Association (NDHIA), whose members own two-thirds of the nine million dairy cows in the United States. NDHIA and iYOTAH have formalized a strategic partnership. They will be working together to deliver value through iYOTAH’s platform to NDHIA’s membership and the industry as a whole.

iYOTAH is also building a set of tools to offer proactive advice and recommendations to farmers. This will be based primarily on machine learning analysis that combines disparate data sets, such as herd data and breeding data. iYOTAH is collaborating with top universities in Agriculture and Data Science, like Purdue and North Carolina State University, to incorporate advanced research models that interpret disparate data and build predictive and prescriptive models for producers. “We’re not just trying to aggregate data, but also apply industry and expert knowledge to incorporate better decision making,” Meza said. iYOTAH is also building data pipelines that will ingest data into Rockset straight from IoT sensors, skipping the S3 staging area, to minimize latency for real-time alerts.

iYOTAH’s current platform built around Rockset is focused on the dairy industry, but will quickly be deployed into other segments such as beef, pork and poultry.

“We have a data pipeline and platform that can be applied for all animal livestock and can have significant impact on the food supply chain as a whole” Meza said.