Tech leaders weigh in on the upside and flipside of generative AI

Obstacles and obstructions

Safety, bias, accuracy, and hallucination continue to be recurring issues.

Jon Cosson, head of IT and CISO at wealth management firm JM Finn, recalls asking ChatGPT for his own biography. The system listed only about 70% of his CV, and simply invented a period at a well-known bank.

“We need to realize where it can be enormously powerful and where it assists us, but be careful we retain human oversight,” he says. “It’s made my life easier because it allows me to write documents and make them richer, but if you rely on this beast it can bite you. We’re using it selectively in tests to see its power, but it’s heavily monitored and we won’t deploy anything if it causes any adverse decision making.”

Medica’s O’Brien issues a caution as well.

“Within healthcare the regulatory environment and the commercial frameworks are years behind the technology,” he says. “This makes it almost impossible to monetize, and, therefore, fund the implementation and usage of the algorithms. This is true across both public and independent sectors. That said, I believe once these barriers are overcome, benefits-led implementation will be swift.”

Coby adds that the immature regulatory and legal structures around using generative AI and large language models (LLM) need to be carefully considered, as does the tendency of current programs to hallucinate. “This is why, at this stage, it’s essential that any use is checked by someone with expert knowledge. Moving from PoCs to mainstream implementation will need to be carefully controlled.”

Ivell adds that generative AI could create unwelcome competitive dynamics.

“As part of our preparation of a generative AI strategy, it’s important to understand where this technology could enable competition or startups to use it to attack our market share with new tools producing faster-to-market and lower-cost products or services,” he says. “So there’s a lot to keep aware of—not just how we may exploit it but also keeping an eye on how it’s starting to be used as a threat.”

And in terms of intellectual property risks, IDC’s Ward-Dutton says oganizations’ own IP can leak into the public domain if they aren’t careful when using public generative AI services. “Some system providers are facing lawsuits because they trained their systems on data and content without getting permission from the original creators,” he says, adding that costs can also be prohibitive because the core technology powering today’s generative AI systems is very expensive to train.

Searching for the sweet spots

There are varying opinions where generative AI will make itself most felt. Collins nominates research and design: “It’s perfectly reasonable the challenges of creating a functional website from scratch should go away, as well as other areas that were already ripe for automation.”

O’Brien adds it’s anything that produces content for consumption by humans, where regulation is light and pricing can fund the algorithm.

IDC’s Ward-Dutton says the analyst’s customer panel points to three main clusters: improving customer and employee experiences; bolstering knowledge management; and accelerating software delivery. In time, he predicts, they’ll be joined by enterprise communication (including contact centres); collaboration and knowledge-sharing; content management; and design, research and creative activities.

Despite being too early to say, Coby believes initial successes will be in enabling humans to be much more productive by using generative AI to produce first drafts and then use them as foundations. “This is likely to be in multiple areas and will require new skills in asking the right queries,” he says.

Ivell concurs regarding areas of content, code generation, and customer support, but says he’s most excited by research opportunities.

“AI can analyze large volumes of data in textual form to create new forms, summaries, and analyses of the data sets,” he says. “It can also provide analysis of large data sets to produce enterprise-level insight previously unavailable such as understanding patterns in behavior and creating insight we can use to build new products.”

JM Finn’s Cosson, an enthusiastic blogger, says text and graphical content using tools such as Midjourney are obvious near-term opportunities.

“It’s already powerful in blog sites and a lot of people will use it as a framework,” he says. “You don’t want to lose the human creative element but you can apply human oversight elements and deliver some outstanding pieces. Where you see downsides are in marketing types and copywriters losing their jobs, but there will be new jobs created.”

A Trojan horse?

Some watchers believe that generative AI can be the trailblazer for wider application of AI and ML. IDC’s Ward-Dutton is particularly enthusiastic.

“In just a few months, generative AI has simultaneously captured the attention, imagination, and trepidation of tech and business leaders across the world,” he says. “We believe generative AI is a trigger technology that will usher in a new era of computing—the Era of AI Everywhere, which will completely change our relationship with data and how we extract value from both structured and unstructured data. The rapid adoption of generative AI moves AI from an emerging software segment in the stack to a lynch-pin technology at the center of a platform transition.”

But CIOs are vocal about the importance of robots working in tandem with people.

“AI works best when it works together with humans,” says Cosson. “The human brain is still worth something. Empathy and humanity are important and we need to work out how AI complements and fuses them together.”