The year’s top 10 enterprise AI trends — so far

“And we may have even underestimated it,” Patience says.

He’s currently tracking 262 gen AI vendors, of which 117 specialize in text generation, and plans to produce a new version of the report in the next six months. Normally, he says, these kinds of reports are refreshed every two years, but this market is moving too quickly for that.

One reason gen AI is getting added faster to enterprise software than previous generations of AI is that it can potentially change the relationship between humans and software, he says.

“It has the ability to let people converse in natural language and get things done,” he says. “Previously, they’d need to code or understand Excel or a query language. Through the use of natural language, you can as a human run complex queries on data sets and other things that are too complex to do on your own.”

Patience has been following this space for more than two decades and says he’s never seen anything like it.

“It’s incredible,” he says. “Our clients, and clients I’ve never spoken to before, all want to know what’s going on. For those who have skills, it’s going to be a force multiplier. For others, it’ll be a bit more of a threat. But it’ll enable people to do higher-value work than they are currently able to do.”

Business process automation

AI has long played a role in RPA, albeit a small one. ML was used for sentiment analysis, and to scan documents, classify images, transcribe recordings, and other specific functions. Then gen AI came out.

“The world has flipped since 2022,” says David McCurdy, chief enterprise architect and CTO at Insight. “We’ve done a number of things with our customers that weren’t in the toolbox 12 months ago. You now have the ability to jump over processes that have existed for years, sometimes decades, because of generative technology.”

One of the best immediate use cases is summarizing documents and extracting information from material, he says.

“This wasn’t possible before,” he says. “Now you can go in and extract a concept, not just a word. It’s transforming some of our workflows.”

Enterprises still aren’t extracting enough value from unstructured data hidden away in documents, though, says Nick Kramer, VP for applied solutions at management consultancy SSA & Company.

“Existing technology just doesn’t surface the most relevant content consistently and easily enough,” he says. “This is where large language models get me really excited. The ability to ingest the corpus of company knowledge offers limitless possibilities.”

AI vendor management

Only the biggest companies are going to build or manage their own AI models, and even those will rely on vendors to provide most of the AI they use. Microsoft, Google, Salesforce—all the major players are all in on AI, so it only makes sense to leverage that. But as gen AI touches more of a company’s data, people, and processes, that vendor selection and management process becomes increasingly important.

“I wouldn’t acquire operational technology that didn’t have ML and AI capabilities,” says Insight’s McCurdy. “If the companies aren’t leveraging AI, and don’t have a roadmap, we won’t buy their software.”

This is one of the ways that enterprises can avoid technical debt, he says, by investing in partners and companies investing in AI. But even when a vendor has AI on the roadmap, or is already building it, there are still risks. “Just like in the early days of the Internet, a lot of companies will come and go,” says Rob Lee, chief curriculum director and faculty lead at the SANS Institute. He’s already seeing this in the cybersecurity space. “At Black Hat, there were at least a hundred companies I saw,” he says. “But do they have something truly sellable?”

One thing buyers have to be careful about is the security measures vendors put in place. With new technology deployments, security often comes as an afterthought. With AI, that would be a big mistake.

“What happens if you upload your data to these AIs?” Lee asks. “You want to experiment, but if someone uploads the wrong spreadsheet to the wrong AI, you have a data breach.”

Trustworthy AI

Last year, as classic AI became increasingly deployed into production, companies began to take the issue of trustworthiness more seriously. They wanted models that were reliable, free of bias, and built on ethical principles. Plus, the AI should be transparent and understandable since people want to know why AIs make the decisions and recommendations it does. Today, trustworthiness is a top priority for everyone from college students trying to get help with their homework, to global leaders looking to avoid an AI apocalypse. Researchers, vendors, consultants, and regulators are working on coming up with guardrails and ethical principles that will govern how AI is trained and deployed.

“We’re still in the early phases of this,” says Donncha Carroll, partner in the revenue growth practice and head of the data science team at Lotis Blue Consulting. “You don’t want to trust a system where you can’t see or audit how it’s operating, especially if it can make decisions that can have consequences. The oversight piece hasn’t been figured out yet.”

Open-source AI

Open source has long been a driver of innovation in the AI space. Many data science tools and base models are open source, or are based heavily on open-source projects. For a few months this year, there were concerns the new space of gen AI would be dominated by tech giants, companies who had the millions of dollars needed to train large language models (LLM), and the data to train them on.

OpenAI’s ChatGPT, Google’s Bard, IBM’s Watson, Anthropic’s Claude, and other major foundation models are proprietary. But in February, Meta released Llama, an open-source LLM licensed for non-commercial use, which quickly became the base for many projects. Then, in July, Meta’s Llama 2 came out, and this time, it was licensed for commercial use. Anyone could use it or modify it, for free, as long as they had fewer than 700 million active daily users. Microsoft quickly pledged to support it on its Azure platform. So did Amazon on AWS. And VMware made it one of the cornerstones of its gen AI stack.

In August, Meta continued releasing models. This time, it was Code Llama, an LLM trained for writing code. Then in September, the UAE’s Technology Innovation Institute released Falcon 180B, the largest open-source model yet. It quickly rose to the top of the Hugging Face open LLM leaderboard, previously dominated by Llama 2 and its variants.

Falcon was also released under a variant of the Apache 2 license, available for commercial use, and works for both natural language generation and code.

The open-source models make it possible for enterprises to deploy customized AI in their own infrastructure, without having to send their data to a cloud provider and offer greater flexibility and lower costs. Some of these open-source models are even small enough to run on desktop computers or mobile devices.

“You’re going to see more of this incredible computation power being distributed on the edge,” says Lotis Blue’s Carroll.

Secure, reliable data infrastructure

Both ML and gen AI depend on data. Over the past 10 years, data has grown to be a company’s most valuable asset, the electricity that powers innovation and value creation. To make all this possible, the data had to be collected, processed, and fed into the systems that needed it in a reliable, efficient, scalable, and secure way. Data warehouses then evolved into data lakes, and then data fabrics and other enterprise-wide data architectures. All that’s going to prove valuable, both as companies continue to expand their traditional AI projects, and for the new gen AI functionality coming online. For many companies, that means public-facing chatbots like ChatGPT aren’t an option because of the lack of enterprise-grade data protection.

“There’s a need to protect the data going into them,” says McCurdy. “That creates an obvious barrier for some use cases until you establish that security perimeter.”

For some, that means running OpenAI’s model or others in private clouds, or even running open-source models on prem, depending on a company’s risk profile. Meanwhile, even after years of effort, a lot of companies still don’t have their data ready for AI. According to S&P Global’s new 2023 Global Trends in AI survey of 1,500 AI practitioners and decision-makers, released in August, the biggest technological challenge to deploying AI is data management. And even though 69% of companies have at least one AI project in production, only 28% have reached enterprise scale.

Accelerating pace of change

Gen AI wouldn’t be possible without the global connectivity afforded by the Internet and the massive amount of information so easily available in digital form, ready to be used as training data. Then there’s cloud computing, SaaS, and APIs, which allow new technologies to be deployed quickly and easily without large upfront integration costs for enterprises. So it’s no surprise the adoption rate of gen AI is faster than that of any technology seen before. But beyond that, gen AI is also a technology that helps to accelerate its own development.

In April, venture capitalist Yohei Nakajima wondered if it was possible to have an “AI founder,” which could run a company autonomously, and asked ChatGPT to build it. It took about three hours total, and ChatGPT wrote the code, the research paper, and a Twitter thread. Nakajima called it “BabyAGI” and it went viral on GitHub. It was an all-purpose agent that could be set to work at any objective, not just starting a company.

“I jokingly asked the autonomous agent to create as many paperclips as possible,” Nakajima wrote in a blog post describing the project. “It found out about the AI paperclip apocalypse and started by generating a safety protocol.”

BabyAGI uses OpenAI’s GPT-4 API, Pinecone vector search, and the LangChain AI framework to figure out what tasks need to be done to achieve an objective, how to prioritize those tasks, and then do them. Similar projects include AutoGPT and AgentGPT.

Another example of AI pulling itself up by its own bootstraps this year was Alpaca, where Stanford University researchers used one of the early Llama models from Meta. This was a raw model that hadn’t undergone reinforcement learning from human feedback—an expensive and time-consuming process. Alpaca took a shortcut, using OpenAI’s text-davinci-003, a close relative of ChatGPT, to generate 52,000 Q&A pairs and used them to train its new chatbot. The whole process cost less than $600, researchers said, $500 of which was spent on the OpenAI API and $100 for computing costs. And when the team tested it, it was comparable to text-davinci-003 in performance. In other words, gen AI models can write new code to improve their own performance, and they can generate data to train the next generation of models.

“The tools and toolkits are changing so quickly,” says Priya Iragavarapu, VP of digital technology services at AArete, a management consulting firm. “Even before leaders and the community are able to read and understand the manual.” This creates challenges for companies trying to plan ahead, she says, since it’s difficult to tell what’s already possible and what’s still in development. “It’s getting hard for leaders to delineate between the two,” she says.

As a result of the fast pace of change, many firms look to build flexible frameworks—ones that allow them to drop in different models as they develop. For example, PricewaterhouseCoopers is not tying itself down to any particular LLM.

“We have a plugin architecture,” says PwC’s Greenstein. “We’ve been helping people build to whatever standards exist, but still have flexibility.”

The company also has people keeping a close eye on the leading edge of development. “With AI, it’s coming so quickly,” he says. “We’re focusing on ‘no regret moves’ like building an LLM-agnostic infrastructure. That’s critical now because the models are leapfrogging each other.”