10 tips for getting started with decision intelligence
For organizations looking to move beyond stale reports, decision intelligence holds promise, giving them the ability to process large amounts of data with a sophisticated mix of tools such as artificial intelligence and machine learning to transform data dashboards and business analytics into more comprehensive decision support platforms.
Successful decision intelligence strategies, however, require an understanding of how organizational decisions are made, as well as a commitment to evaluate outcomes and manage and improve the decision-making process with feedback.
“It’s not a technology,” says Gartner analyst Erick Brethenoux. “It’s a discipline made of many different technologies.”
Decision intelligence is one of the top strategic technology trends for 2022, according to the analyst firm, with more than a third of large organizations expected to be practicing the discipline by 2023.
The trend is brewing at a time when organizations need to make decisions faster than ever — and at a scale not yet seen. Decision intelligence helps provide an automated way to make decisions, which in turn can help companies stay competitive and meet market demands, Brethenoux says.
But that takes a deep understanding of the decision-making process, the risks and rewards of each decision, the acceptable margin of error, and the ability to figure how confident you should be in any decision offered by your automated decision processes.
Here are some tips to help you do all of that.
1. Start with low-hanging fruit
It helps to start with a process that is extremely well-defined, low-risk, and has a large collection of examples. Many companies have such processes already in place, and not all of them are fully automated yet.
Companies too busy with the day-to-day might not notice that they’re missing these opportunities, says Ray Wang, principal analyst and founder at Constellation Research. “Then they start wondering why competitors are doing better but by the time they’re doing that, it’s too late.”
Even when a process has already been automated, adding more factors to the decision engine may improve accuracy, he says. “The more attributes you have, the more likely those things haven’t been correlated,” he says.
For example, a risk scoring decision might be improved by considering the time of day, or the user’s location.
The key takeaway, though, is that decision intelligence isn’t a once-and-done process. You must continually tweak your approach based on feedback.
2. Let new data also be your guide
The more often a process is repeated, and the clearer the results, the more opportunities a company will have to improve it.
LexisNexis, for example, uses its ThreatMetrix product to make 300 million fraud-related decisions a day, but the decision’s aren’t 100% perfect.
“We are in the spectrum of making many decisions across a huge dataset that are not life-threatening if we get them wrong,” says Matthias Baumhof, CTO at LexisNexis Risk Solutions. “But they offer huge value to the customers if we get them 99% right.”
LexisNexis uses machine learning algorithms to sort transactions into behavioral profiles to predict whether any particular transaction is fraudulent, or suspicious. There’s historic data, for the initial training set, as well as ongoing training.
“If a current transaction is confirmed to be a fraud after a few days, and they share that with us, we can learn from the confirmed fraudulent behavior,” he says, noting that for anyone looking to make the most of decision intelligence, now that behavior patterns change. “A certain amount of learning is always business as usual. If you don’t learn, you actually fall behind.”
3. Tweak your algorithms
Risk scoring traditionally involved a series of if-then decisions. If a transaction was over a certain amount, or outside the user’s home area, or with a new merchant, it would be flagged for review. But as the decisions get more complicated, it’s hard for if-then systems to keep up.
“Even when customers have tuned their rules for years with fraud analysts who know the space, we come in with machine learning models and beat them,” says Baumhof. “But you can run them in parallel and get the best of both worlds.”
Current machine learning systems can make decisions as fast as traditional rules-based systems. But six years ago, when LexisNexis began to invest in machine learning as a replacement for rules-based systems, the company started with a linear regression model. An example of a linear fraud relationship might be that the further away from home a purchase is made, the more likely it might be fraudulent.
But this approach proved too simple, incapable of detecting non-linear relationships that don’t go smoothly in one direction. For example, transactions that are unusually small might be a sign of fraud, with criminals testing out a card number or account to be sure that it works. For this, the company has turned to gradient machine learning.
“We have made the best strides with gradient boosting trees,” Baumhof says. “It provides high accuracy with short latency.”
This new approach has been tested over the past year and will be rolled out into production in the second quarter of this year, he says. The company next plans to explore new technologies, such as deep learning, Baumhof says. “That’s definitely something on the radar, to see if they can beat the current models that we have.”
So, in addition to incorporating new data into your decision intelligence strategy, rethinking the underlying algorithms can also help increase the quality of your results.
4. Augment complex processes — especially for data collection
When decision steps are less clear, outcomes more nebulous, or there are bigger risks to getting decisions wrong, intelligent systems might not be able to replace all the decision-making, but they might be able to augment it.
For example, LexisNexis uses machine learning to analyze court documents, says Baumhof, nothing that, for example, a plea might need to be written in a particular way to get a positive reaction from certain judges.
Or in analyzing contracts with third parties, which, instead of having millions of relevant examples for training, might offer only thousands, or hundreds, of examples. In those cases, “the machine learning would just give you a proposal,” he says. “But a human being would do the final version of it.”
The automation component of decision intelligence can come in during the data collection phase of decision-making, Constellation’s Wang points out. It doesn’t have to come up with the final conclusions, and can also be used to create reports or generate trends and correlations.
The old way, of manually collecting data and producing reports, isn’t a good idea today, Wang says. “You want that information at machine-scale and right now.”
5. Separate the good from the lucky
With smaller data sets, it can be very difficult to tell whether a decision was good but, through sheer luck, led to a bad outcome. Or if a decision was bad, but luck intervened and things worked out, anyway.
“The quality of outcomes and the quality of decisions are not the same thing,” says Amaresh Tripathy, global leader of analytics at Genpact. “Sometimes you have a great set of cards and make the right decisions but you still lose.”
Unfortunately, when it comes to complicated and infrequent decisions, businesses don’t usually have mechanisms in place to measure this. But solving this issue isn’t about technology, Tripathy says.
“The first step is to formalize a decision-making process in the organization, and only then can you think about adding software to support that process,” he says.
Collecting the outcomes of these decisions and linking them back to the decision-making process, however, is challenging. Companies in the marketing space are the most adept at this right now, Tripathy says. “They regularly do A-B testing, changing the colors and the fonts,” he says. “Or they change the menu items. They test a lot.”
In life sciences, a similar process goes into drug discovery and vaccine development, he adds. In human resources as well, companies can examine their decision-making processes and look at the results.
“With hiring, the outcomes are relatively clear,” he says. “You can see the hires’ performance. The hardest part of the business is when the outcomes aren’t very clear.”
6. Watch out for biased data
Decisions are only as good as the data they’re based on. If a company’s history is problematic, then a training set based on that history can inherit the same problems.
For example, a company that in the past only hired white men with Ivy League educations might end up with a hiring recommendation system that only recommends white men with Ivy League degrees. But that’s only part of the story.
People are also inherently biased, says Brad Stone, CIO at Booz Allen Hamilton. And they will seek out data that supports their biases. “If we think we need more recruiters, we will find data that will prove that we need more recruiters,” he says. “And if we think that we need more business operations folks, we can find data that supports this as well.”
And when people look at data, they look at it through the lens of their experience with it, he says, which may lead to flawed conclusions. “The pandemic in particular has taught us that you can’t just trust the past to predict the future,” he says.
The solution, he says, is to provide the right guardrails for decision making. “The successful businesses and missions of the future will be able to learn from the past while managing that bias,” he says.
7. When the AI works, trust the AI
Sometimes, data-driven recommendations fly in the face of all instincts, and not understanding how the technology works can set a company back by years.
Michael Feindt, strategic advisor and founder at Blue Yonder, a supply chain management technology company, has seen many companies struggle to accept that their instincts might not be accurate. For example, ordering fresh food at a grocery store is an asymmetric cost function, he says. If there’s too little, customers will be disappointed, but if there’s too much, then the food will spoil. The costs are not equal.
The same principle comes into play with any product with a limited lifespan, such as seasonal fashions in the clothing industry, as human brains are not wired to calculate the risks correctly.
For example, one German department store chain Feindt worked with started using AI for its ordering six or seven years ago — and gave up using it after three years. “Both the employees and the senior managers did not understand it,” he says. “The managers are not mathematicians. They are convinced that they are right because they’ve always done it that way.”
So each year at Christmas, store managers panic at the thought of not having enough products. “And they buy like hell,” he says. “Two weeks before Christmas, the CEO says, ‘We have to have more meat and more cookies. Order more, order more. Whatever you want to order, add 50%.’ The software already knows it’s Christmas. This is exactly where AI is very good. It can predict these things. But because of the fear that they don’t have enough, they add 50%. And after Christmas, they throw away that 50%. It cost them more than a million Euros.”
The solution, he says, is to have at least one person involved in these kinds of decisions who understands how the analytics work, at least one qualitative person who has the trust of management.
8. Use synthetic data
In some cases, the lack of training data can be compensated for with synthetic data.
Synthetic data, which is artificially generated information that is accurately modeled for use in place of real historic data, can provide machine learning systems with more fuel to function. Use of it can enable companies to apply automated intelligence to many more cases, says Gartner’s Brethenoux.
It can also enable companies to train for black swan events or unusual scenarios. “Synthetic data is becoming one of those techniques that helps us out,” he says.
According to Gartner analyst Svetlana Sicular, by 2024, 60% of the data used for the development of AI and analytics solutions will be synthetically generated, up from 1% in 2021.
9. Use tabletop exercises to simulate various outcomes
In many situations, making the right decision is an impossibility, as too many external factors have undue influence on the outcome. A new COVID wave, another tanker stuck in a canal, a regional drought, a war breaking out — any of these could have a dramatic impact on a business but are completely unpredictable.
That doesn’t mean companies are powerless. Instead, they can run simulations to prepare them for multiple scenarios. And they can collect all the data, to make as informed a decision as possible.
But there’s a limit to how far data and analysis can take you. “I participated in many acquisition decisions,” says Gartner’s Brethenoux. “Sometimes the CEOs fall in love with a deal. It’s fun and exciting. And sometimes they forget the basic principles.”
But with big decisions, a lot of factors come into play, he says. One of those factors could be whether the CEO can rally people against all odds. “Sometimes they’re visionary,” he says. “They make it work purely by charisma, nothing to do with the value of the deal. If he or she is that kind of person, we can ignore the data because the CEO can make it work.”
10. Start small and learn
The important thing is to consider decision intelligence as a viable possibility, and to test it out. “You can start small,” Gartner’s Brethenoux says. “In fact, many companies are already doing decision intelligence without calling it decision intelligence.”
That includes online retailers that have recommendation engines, for example. But they’re not always taking advantage of all the perspectives that decision intelligence calls for, he says.
“When people act on a recommendation, there’s a transaction,” he says. “But when they don’t buy, very few organizations analyze that. They don’t analyze the transactions that don’t happen. But why didn’t people buy? Was it the wrong product, wrong price, wrong time?”
With a decision intelligence mindset, those non-transactions should also be analyzed, he says.
“You can do decision intelligence today,” Brethenoux says. “Just add a little bit to your investment, and do something.”
