NeuReality announces power-saving AI appliance

“Systems built with CPUs simply can’t support and scale the number of accelerators that are put in the system,” he said. “The CPU becomes a data-moving machine, and unfortunately, CPUs were built to compute, not to move a ton of data back and forth. There are a number of reasons why the CPU architecture basically becomes a bottleneck.”

The appliance takes over for the CPU, greatly enhancing the throughput and scalability of the GPUs. This allows GPUs to run much faster and at greater utilization than with CPUs. When paired with Qualcomm’s AI 100 Ultra, NR1-S achieves up to 90% cost savings across various AI data types, such as image, audio and text. Along with the cost saving, the NR1-S shows up to 15 times better energy efficiency compared to traditional CPU-centric systems. Unlike traditional CPU-centric systems, NR1-S can ensure 100% utilization of the integrated AI accelerators without performance drop-offs or delays observed in today’s CPU-reliant systems, the vendor claims.

The tests also measured energy consumption for audio processing per watt. In a voice-to-text test, the NR1-S was able to convert seven seconds of audio using the same amount of power as 0.7 seconds in traditional CPU-centric systems. This translates to a 10-fold increase in performance for the energy used, according to NeuReality. Another audio test showed NR1-S cutting the cost of processing 1 million audio seconds from 43 cents to only 5 cents.