At a loss for data project ROI? Evaluate it like a product

In 2006, British mathematician Clive Humby proclaimed, “Data is the new oil.”

Humby had bona fides to make that claim. He and his wife, Edwinna Dunn, own Dunnhumby, a global customer data science company that helped Tesco create its Clubcard, the world’s first supermarket loyalty card. The program gave the British retailer unprecedented insight into its customers and their buying habits.

Today, Doug Laney, innovation fellow of data and analytics strategy at West Monroe, disputes Humby’s assertion on a technicality: “When you use a drop of oil, you can only use it one way at a time,” Laney says. “When you do use it, it gets used up. And when you use a drop of oil, it doesn’t create more oil. Information or data is very different.”

Data doesn’t deplete when you use it. It can be used multiple ways simultaneously. And using data typically creates more data.

“Data is what economists would call a non-rival risk, non-depleting progenitor of assets,” Laney says.

Whether or not the oil metaphor is apt, what is not in dispute is that data has the potential to unlock enormous value for organizations that can leverage it properly. Laney, a former distinguished VP analyst at Gartner, studied how companies used their data when he was with the research firm.

“We found that companies that treat data more as an asset have a market-to-book value ratio that’s nearly two times higher than the market average. And companies that sell data products or data derivatives of some kind have a 3x market-to-book value ratio,” he says.

Laney notes that a more recent Gartner study of chief data officer success found CDOs were 3.5 times more likely to achieve success in their role when they met data monetization initiatives, versus only 1.7 times more likely when they demonstrated ROI on their BI or data analytics investments.

Increasing numbers of West Monroe clients are asking the firm to help them through data monetization exercises: ideation, testing the feasibility of components, and laying out a roadmap for creating data products, Laney says.

That doesn’t mean all data products need to be focused on selling data. Many valuable data products support organizations’ operations. Supply chain analytics and digital twins are some key examples.

“Data monetization can take a lot of different forms and formats,” Laney says. “The industry is starting to recognize that. They’re starting to get outside their own heads when it comes to thinking about data monetization as being more than just selling data.”

So, while not every enterprise will productize their data assets for consumption outside the organization, chief data officers would be wise to evaluate each data initiative as if it were, by evaluating its costs of creation and maintenance and coming up with a framework for ascertaining whether the revenue it generates, however indirectly, has been worth the effort. 

Establishing the net value of data operations

Of course, accurately valuing data assets is easier said than done. If a data product is sold directly, the calculation is relatively simple. But what if it is a support tool? How do you value it then?

That question recently sent data intelligence company Collibra on a journey to create a process for assessing the value of its data products, including tools that don’t directly create revenue themselves.

“At Collibra, data is treated as a product, not just an asset,” says Jay Militscher, head of the data office at Collibra. “It could be a dataset, an ML model, or a report. Product-based thinking means that there’s an owner in the business, managing it strategically with an ROI attitude. Data valuation is all about converting data to value in a deliberate and strategic way. Value comes in the form of either directly selling data and analytics right into the market or changing the economics of how you do work processes: efficiency, speed, identifying an opportunity, stuff like that. All companies, we believe, have to address this data monetization and valuation topic at some point, if they’re not already.”

Laney agrees, noting that many companies have struggled with their data monetization efforts over the years because they have failed to take a product management approach.

“[Those companies] have been kind of dabbling in it or are doing one-off things,” he says. “The companies that are doing this well are taking a really defined product management approach to conceiving and planning new ways to generate new value streams from data, to identifying and developing markets among partners and others throughout their extended business ecosystem, coordinating properly with IT, marketing, finance, legal, and other product lines and management groups to execute on these data monetization initiatives.”

Collibra has created an eight-step process for creating data products to help it focus on the product management approach. The process begins with identifying a business need and defining the data product owner and culminates with monitoring the data product in production.

One of the first big projects Militscher’s team undertook at Collibra was the Data Intelligence Usage Dashboard, which helps Collibra pre-sales engineers understand how customer adoption evolves during a trial. Used correctly, it can improve a customer’s user experience and satisfaction during a proof-of-concept, but while the dashboard seemed to have a lot of success, the monitoring step of the process showed that usage was not equally spread across all pre-sales engineers. Some were using it heavily and others barely touched it.

“It made us wonder, Is this data product really valuable? Was it worth the cost and the effort that we put in this? Imagine if we could estimate those revenues and costs at the start of the project and reevaluate those estimations with real numbers one year later,” says Alexandre t’Kint, data scientist at Collibra.

With the help of intern Sarvenaz Rahmati, t’Kint created a process to evaluate the costs that went into creating the Data Intelligence Usage Dashboard, the revenue (both tangible and intangible) that it generates, and ultimately what the dashboard’s net value is.

Collibra

First, t’Kint and Rahmati had to identify all the resources the Data Intelligence Usage Dashboard uses to determine its costs. The dashboard stores raw data in Amazon S3, uses Amazon EC2 for cloud computing, and Amazon Redshift for data warehousing. They then needed to determine what they call “the creation cost,” the cost for all people involved in creating the Data Intelligence Usage Dashboard, which included a data scientist, a data engineer, a pre-sales engineer, and two senior managers.

Maintenance cost, what it takes to modify a data product after delivery, is a third important factor in a product’s overall cost. t’Kint notes that the Data Intelligence Usage Dashboard has rarely failed, so its maintenance cost has been minimal.

Finally, t’Kint’s cost-assessment process required ascertaining any license costs involved. The dashboard required Collibra buy a Tableau license for all the people who would create or view the dashboard.

With costs assessed, t’Kint and Rahmati then turned to evaluate revenue. This is challenging if a data product is not a direct revenue generator. For the dashboard project, t’Kint says they considered looking at the number of views on the data product or calculating the value of deals closed per user of the data product. Ultimately, they decided to survey the sales engineers to determine how much they valued the dashboard. Based on that survey, they estimated that 4% of the revenue from closed deals over a period of six months could be attributed to the existence of the dashboard.

A framework for data project ROI

According to t’Kint, the process of establishing a net value of operations for the Data Intelligence Usage Dashboard helped Collibra determine that, yes, the project had created net positive value. It also helped the team identify next steps to increase that value.

“The value was already positive, but then we wanted the value to be as great as possible,” t’Kint says. “Our follow-up project was trying to increase the usage of these insights even more.”

The data office worked with Collibra’s IT department to embed the dashboard’s insights into Salesforce, where the company’s pre-sales engineers live all day.

“If you think about the consumption of data products, in the past, people had to first go to the dashboard tool, then consume the data, take a decision and finally perform an action,” t’Kint says. “By embedding these insights where people live, you remove one of the three steps and bring the insights to them.”

Ultimately, t’Kint says this process will help Collibra be even more deliberate and strategic about the creation of new data products over time. He recommends other data leaders adopt a similar approach.

“You need to have a sense of ROI, beginning with a sense of what it costs to product it and being deliberate about measuring what you expect to reap from it is really the starting point,” he says. “Your measures might include how the product is going to improve the way you work as the first step. You can interview users about how much quicker their workflow is because of the data products. Estimating the value upfront is going to be the hardest part, but don’t block on that.”