AI workloads: There’s room for Nvidia competition in the AI market

In the data center, he expects to see more traditional data center servers running AI workloads as they move toward inference-based workloads, as well as fine tuning and RAG optimizing of existing models. Inferencing is much less process-intensive than training and can be done on traditional CPUs instead of more expensive GPUs.

This is opening up an opportunity for AI as a service, provided by major cloud service providers, where a company can have the AI training done on the expensive hardware without having to make a major capital investment in hardware they only need once and then do the updates or inferencing with their own gear.

“It’s also likely that as newer, more efficient modeling methods are developed, they will increasingly be run on traditional servers, both from a cost/performance advantage perspective as well as for greater compute availability. This will benefit the traditional players who have a well-established data center business,” Gold wrote.

On the edge, Gold expects the vast majority of AI workloads to migrate to edge-based systems over the next two or three years. What qualifies as the edge is a wide range of systems and processing capabilities – from small internal processing in sensor arrays to heavy machinery, autonomous vehicles and medical diagnostics, just to name a few.

Gold predicts that open-source platforms and development environments will play a key role in this space as opposed to proprietary solutions like Nvidia’s CUDA. “Open and compatible ecosystems like Arm and x86 from will have significant advantages as they create compatibility from small to large computing needs. They allow up scaling or down scaling as the processing requires as well as ease of porting solutions and reuse,” he wrote.

The IoT space has a lot of overlap with edge computing, and therefore there is a need for an open ecosystem to provide scalable solutions, much like the edge. It’s just that with IOT, the devices tend to be smaller and lower power, but there are plenty of players in that field.