Why enterprise CIOs need to plan for Microsoft gen AI

Enterprises with strong experience in open source may look to open foundation models as an option to reduce costs, but Curran cautions against equating open weight models with the more familiar open source ecosystem. He predicts enterprises will adopt them though, including using them in curated environments provided like Azure.

“I’ve seen a lot of interest in the open source models, but not many in production,” says Boyd, although customers are starting to use small models like Microsoft’s own Phi series. “But it’s largely early days. I haven’t seen mass adoption.”

Beyond simplifying setting up and running open weight models, using them on a platform like Azure has an added benefit: Microsoft’s model content safety service is “on and integrated by default with Azure Open AI Services, but it’s also on by default with all our open source models as well,” he adds.

After the excitement and experimentation of last year, CIOs are more deliberate about how they implement gen AI, making familiar ROI decisions, and often starting with customer support. “It’s a cost most organizations have but don’t like paying for, yet they still want to provide a quality experience,” he says. Reduced call times and escalations are obvious benefits as well.

These more vertical, task specific, integrated gen AI offerings may contribute more than generalist productivity copilots because people won’t need to find uses and then remember to include them in their workflow. But the most popular copilots can perform strongly: Virgin Atlantic, for instance, reports efficiency gains of 14 minutes per day.

But not all copilots necessarily provide the same value. Curran suggests security copilots may not provide significant extra value on top of existing tools in Microsoft Defender, at least without extra training. But the Excel Copilot was a surprise hit at Virgin Atlantic. “People absolutely love that Copilot will automatically tell you if you have data inconsistencies in the way you’re filling out forms,” says Walker. He describes how Copilot can warn if, say, you’re adding a duplicate filter in lower case instead of upper case, and fix it. “It’s like having someone look over your shoulder as you’re doing it.”

The Teams Copilot to summarize meetings and provide next steps is almost universally popular, too. “You get into a room with 15 people and you’re not focusing on who’s taking the minutes or whether you need to be clear enough in allocation and make sure everyone understands what the output is,” says Walker. “You’re focusing on the meeting itself, and you’re more present in the room because you know Copilot is behind you recording and transcribing.”

Even this pre-built Copilot needs preparation before enabling. Multilingual organizations where staff speak in both their native languages and a common language like English or Mandarin will need to monitor quality of transcriptions and translations more carefully. And if recording meetings isn’t already common in the organization, CIOs need to consult with legal and data protection teams on retention, auditing, and deletion policies because of potential issues around discovery.

A data leakage plan helps here too. “As soon as you record something, it becomes a form of data and needs classification and a place in the organization,” Walker says. “But equally, you need to know whether it’s appropriate to create that data in the first place.”

While CIOs need to maintain financial discipline and track usage of gen APIs with the now familiar ‘pay as you go’ model, especially with September budgeting season looming, they also need to play a long game warns Mickey North Rizza, group VP, Enterprise Software at IDC. “It’s going to cost you a lot of money,” she says. “CIOs may complain they’re not getting enough out of it, but the first time you got an iPhone, nobody knew what to do with it.”

Whether used as an assistant, advisor, or an agent, she expects gen AI’s optimized access to information to reduce multi-step business processes to real-time systems with far fewer steps. But experimentation to achieve significant results takes time.

In the meantime, Boyd notes, OpenAI prices have significantly reduced. “In the year and a half since Azure OpenAI Services has been available, ChatGPT 4 has fallen by 12 times while being six times faster,” he says.

Make training specific

Phased deployments aren’t just about cost, security, or compliance concerns, but capturing the right feedback to manage them well and support users properly. Training is key, even when considering gen AI skills in hiring, as is being willing to accept the simplified processes gen AI can produce. Troublingly, there’s a considerable disconnect between what leaders think their employees are ready for with gen AI and what staff feel prepared for.

Forrester found 59% of leaders believe they’ve given staff sufficient training, but only 45% of employees say they’ve had any formal training. The most successful training covers not just staff roles but their workflows. There’s enormous enthusiasm for gen AI but engagement quickly drops off if they don’t have the time to explore it and learn how to get useful results for their work, Wong says.

“If you don’t use the technology to fundamentally rethink processes, and you just layer more AI work over existing processes, you don’t get the best benefit out of it,” he says. “You have to rethink the underlying processes, and have training and ongoing education because these technologies are moving very quickly. The paradox is employees still want it despite the fact it’s hard for them to ingrain generative AI into their work routines, and that in some cases it’s underwhelming based on their expectations.”

CIOs may then want to consider how organizations adopt low code tools, where encouraging bottom up enthusiasm, experimentation, and sharing of growing expertise helps spread usage across the business. Both Microsoft and Virgin Atlantic report good results from structured training that includes time to experiment. Walker refers to “guided play sessions” and users were encouraged to share what worked with their peers. “They can go out as trusted users into the environment and say to people this isn’t scary,” he says.

CIOs should also remember gen AI is just one of many changes organizations are asking staff to absorb. The rate of change enterprise workers are expected to adapt to is up to three times what it was in 2010, Curran warns. “Businesses have not increased their ability to support those changes with the same speed,” he says. Adding resources to support employees through these changes will be as important for succeeding with gen AI as getting the technology right.

That includes IT teams themselves, who need to prepare for gen AI to continue developing at this speed. Vladimirskiy passes on Microsoft’s advice to software partners creating their own gen AI products. “Everyone should have the expectation that by the time you build something, you’re going to have to scrap it and start again,” he says. “The value for companies is maybe not so much the outcome of the product they’re building, but the creation of the expertise within the organization, to be able to leverage it in the future when AI becomes much more capable than it is today.”