It’s a new dawn of AI-powered knowledge management

Open is Better

Just as the web is built on open-source software and protocols, it’s likely that many enterprise AI initiatives will be built on open-source models such as LLaMA and freely available techniques such as LoRa. According to a recently leaked Google memo,

“The barrier to entry for training and experimentation has dropped from the total output of a major research organization to one person, an evening, and a beefy laptop.” 

These barriers to entry will only become lower and the results better, allowing startups and enterprises to build niche models focused on the specific needs of businesses and workflows.

From GenAI to SynthAI

Central to these developments is the move from AI systems that create new content based on simple prompts to models that are trained on an enterprise’s internal data and programmed to generate usable insights and recommendations. 

LLMs such as ChatGPT produce often believable results, but it’s not clear how the data fed into the model was used and whether the answers it gives are true or hallucinations. The recent case of a New York lawyer using ChatGPT to generate court filings and listing presumably historical cases to support his client’s claim showed the dangers of relying on GenAI outputs: despite looking like genuine evidence, six of the cases listed never took place.

Silicon Valley venture capital firm A16Z recently outlined its belief that the future for AI in the workplace was not necessarily LLMs like ChatGPT, but more focused models designed to address specific business needs. They refer to this as SynthAI where the models are trained on proprietary data sets and optimized for discrete purposes, such as resolving customer support issues, summarizing market research results and creating personalized marketing emails.

Applying the SynthAI approach to better managing a firm’s data assets is a natural evolution of this next stage in the AI revolution. Consulting firm BCG has adopted this approach to 50 years worth of their archives, largely reports, presentations and data collected from surveys and client engagements. Previously, employees could only search these files via a keyword search and then read through each document to check its relevance. Now the system provides usable answers to questions.

The knowledge management dream is becoming a reality.