Nvidia says NeMo microservices now generally available

He also detailed how Nvidia partners are leveraging NeMo microservices in their AI agent platforms. AT&T, for example, is using agentic AI to support its call centers. Working with Arize and Quantiphi, AT&T has built an AI agent that leverages NeMo microservices to process its knowledge base of nearly 10,000 documents, which are refreshed weekly. Conway noted that NeMo microservices helped improve the agent’s accuracy by 40%, which also lowered the compute overhead. Then there’s BlackRock, which is leveraging NeMo microservices for agentic AI capabilities in its Aladdin tech platform, unifying the company’s investment management process through a common data language. Cisco’s Outshift team, working with partner Galileo, is also using NeMo microservices for a coding assistant, resulting in 40% fewer tool selection errors and up to 10 times faster response times.

Conway further added that NeMo microservices support open models including Llama, the Microsoft Phi family of small language models, Google Gemma, Mistral, and Llama Nemotron Ultra. Meta offers new connectors for Meta Llamastack, and AI software providers including Cloudera, Datadog, Dataiku, DataRobot, DataStax, SuperAnnotate, and Weights & Biases — have integrated NeMo microservices into their platforms.