- Upgrade to Microsoft Office Pro and Windows 11 Pro with this bundle for 87% off
- Get 3 months of Xbox Game Pass Ultimate for 28% off
- Buy a Microsoft Project Pro or Microsoft Visio Pro license for just $18 with this deal
- How I optimized the cheapest 98-inch TV available to look and sound incredible (and it's $1,000 off)
- The best blood pressure watches of 2024
AI requirements exceed infrastructure capabilities for many IT teams, study finds
As adoption of artificial intelligence (AI) technology accelerates, IT organizations are concerned that their existing infrastructure isn’t powerful enough to keep up.
AI hardware – especially training hardware – is becoming more and more power hungry, according to Equinix, which just released its 2023 Global Tech Trends Survey.
The power draw from traditional racks in a data center is between 5 kW and 10 kW per rack. But, increasingly, newer generations of GPU-based racks are pushing power draws north of 30 kW per rack, and in some cases as high as 72 kW per rack, according to Kaladhar Voruganti, senior technologist at Equinix. “So, definitely, it’s very hard to host this type of infrastructure in private data centers,” he said.
In its research, Equinix found that 85% of the 2,900 IT decision-makers polled are already using AI or planning to use it across multiple key functions. Organizations are most likely to be using AI, or planning to do so, in IT operations (85%), cybersecurity (81%), and customer experience (79%).
However, 42% of IT leaders surveyed believe their IT infrastructure is not fully prepared to meet the demands of AI technology.
Power capacity isn’t the only concern. Cooling, too, is impacted by AI’s climbing power demands.
Traditional air cooling involving heatsinks and fans only works effectively until roughly 30 kW per rack. After that, the fans simply can’t keep up, and liquid cooling is required. Most data centers are not equipped for liquid cooling and would have to be retrofitted at considerable expense.
Along with hardware shortcomings, many companies are dealing with a shortage of skilled IT professionals. There has always been a shortage of people with analytics skills, for example, and that has only been exacerbated with the explosion in popularity and use of AI.
AI as a service
Companies lacking in the proper hardware to do AI training have two options: make a massive investment in hardware or turn to cloud service providers for AI-as-a-service, which most of the top cloud service providers now offer. Rather than make the million-dollar investment in hardware, an enterprise could upload the data to be processed and the cloud service provider can do the heavy lifting. The enterprise could take the trained data models back when the processing is done.
Customers often will opt for end-to-end solutions from AI vendors in the cloud, especially initially, “because they make it easy for the customers with a simple button,” Voruganti said. But variable cloud costs – which enterprises incur with each read or write to cloud-based data, or with every data extraction, for example – may cause IT teams to reconsider that approach.
Voruganti said he’s seeing companies choose to place foundation models with different cloud service providers based on their areas of expertise. So, one provider might offer strong vision-based models, while another might have better general language or large language models.
“There is increasingly desire for people to leverage these models from different clouds,” Voruganti said. “And [if they can maintain their own control over their data], like a neutral location, then they can bring the model to where the data is, and then customize it.”
Copyright © 2023 IDG Communications, Inc.