Enterprises are struggling with what to do with Gen AI, say venture capitalists

This year, there’s been a huge enterprise investment in artificial intelligence (AI), nearing $14 billion. However, a significant proportion of companies are unsure what they’re doing with the technology, according to a survey of businesses by venture capital firm Menlo Ventures.

“More than a third of our survey respondents do not have a clear vision for how generative AI will be implemented across their organizations,” write the authors of the report, Menlo Ventures partners Tim Tully and Joff Redfern, and investor Derek Xiao, who used the help of Anthropic’s Claude Sonnet 3.5 large language model (LLM) to compile the report. 

Also: AWS says its AI data centers just got even more efficient – here’s how

The full report, 2024: The State of Generative AI in the Enterprise, can be read on the Menlo Ventures website. The survey was conducted in September and October and is based on responses from 600 IT decision-makers.

The report is the latest output from Tully, Redfern, and Xiao, who also offered a perspective on AI agents in September. 

Also: The journey to fully autonomous AI agents and the venture capitalists funding them

The authors suggest the uncertainty around generative AI (Gen AI) indicates that “we’re still in the early stages of a large-scale transformation”. 

Indeed, the lack of clarity on AI strategy is just one element of an otherwise very positive piece. Leaving aside spending on AI chips from Nvidia and others, spending on “foundation models, model training + deployment, AI-specific data infrastructure, and new generative AI applications” totaled $13.8 billion in 2024, the authors relate, more than six times as much as 2023’s total ($2.3bn). 

“This spike in spending reflects a wave of organizational optimism,” the authors write. “72% of decision-makers anticipate broader adoption of generative AI tools in the near future.”

The biggest single category of AI spending by those enterprises is foundation models, the LLMs developed by Anthroptic, OpenAI, and others, which soared from $1bn in 2023 to $6.8 billion this year. The smallest spending was on data and infrastructure, at $400 million.

Also: Snowflake customers eke out early gains from Gen AI applications

However, the biggest single increase is for AI applications, which rose eight-fold to $4.6 billion. That figure includes three categories: vertical AI, departmental AI, and horizontal AI.

The application category is “heating up”, the researchers write. 

“While foundation model investments still dominate enterprise generative AI spend, the application layer is now growing faster,” they write, “benefiting from coalescing design patterns at the infrastructure level. Companies are creating substantial value by using these tools to optimize workflows across sectors, paving the way for broader innovation.”

The dominant use cases, by prominence, include code generation via code copilots, including Microsoft’s GitHub Copilot, currently on course to reach $300 million in annual revenue. Next are support chatbots, followed by enterprise search and retrieval, and automatically generated meeting summaries. 

Also: AI isn’t hitting a wall, it’s just getting too smart for benchmarks, says Anthropic

Menlo has a direct financial interest in AI spending, as the firm backs many startups in the area, including Anthropic and vector database maker Pinecone.

In fact, Anthropic is gaining ground against OpenAI, the authors relate, winning converts from GPT to Claude. 

“Among closed-source models, OpenAI’s early mover advantage has eroded somewhat, with enterprise market share dropping from 50% to 34%,” they relate. “The primary beneficiary has been Anthropic, which doubled its enterprise presence from 12% to 24% as some enterprises switched from GPT-4 to Claude 3.5 Sonnet when the new model became state-of-the-art.”

The most forward-looking part of the report covers what Tully, Redfern, and Xiao refer to as the “Modern AI Stack”, layers of infrastructure technology used to build applications. 

The researchers report that “enterprises [are] coalescing around the core building blocks that comprise the runtime architectures of most production AI systems.” 

That approach includes the foundation models, data services riding above them, such as Pinecone, software development frameworks for orchestrating AI agents, such as LangChain, and, at the very top, integration tools, such as those from Composio.

Also: How LangChain turns Gen AI into a genuinely useful assistant

The report offers three predictions for the year ahead.

First, AI agents are poised to “disrupt” the $400bn enterprise software market, led by platforms such as Clay and Forge, “tackling complex, multi-step tasks that go beyond the capabilities of current systems focused on content generation and knowledge retrieval.”

Second, established software firms could be disrupted just like textbook seller Chegg and IT discussions firm Stack Overflow have been. “IT outsourcing firms like Cognizant and legacy automation players like UiPath should brace for AI-native challengers moving into their market. Over time, even software giants like Salesforce and Autodesk will face AI-native challengers,” write Tully, Redfern, and Xiao. 

Third, there will be “a massive talent drought” as AI systems become more prevalent, running against a lack of data scientists and subject domain experts. “Brace for soaring competition and 2-3x salary premiums for already well-paid AI-skilled enterprise architects,” the researchers predict.