Microsoft can’t keep up with demand for AI in the cloud — for now

With immature technologies such as generative AI, Microsoft and its competitors face challenges accurately forecasting changes in demand, according to digital services firm West Monroe’s lead of high-tech and software practice, Dhaval Moogimane. While occasional discrepancies between capacity and demand may persist, it is unlikely to manifest as a protracted or systemic issue that would lead to price hikes, Moogimane said.

Instead, said Shimmin, Microsoft and other hyperscalers will likely resort to other tactics to manage demand, such as downgrading response times for customers paying less or making use of batch inferencing, a process in which predictions are made, stored, and later presented on request. This can be more efficient than online or dynamic inferencing, where predictions are generated in real time.

Batch inferencing, especially in support of API calls is rapidly becoming “a thing” among model hosting providers according, to Shimmin.

Customers shouldn’t be surprised at imbalances between demand and supply in cloud computing, according to IDC analyst Rijo George Thomas: They’re not new and enterprises have been complaining about them since the beginning of the Covid pandemic. “IDC’s Wave surveys have revealed that supply chain constraints were one of the top concerns, at least for Asia-Pacific IT leaders, affecting their tech strategies and budgets,” Thomas said.