Red Hat bets big on AI with its Neural Magic acquisition

Red Hat, the IBM-owned open-source software giant, has completed its acquisition of Neural Magic, a pioneering artificial intelligence (AI) optimization startup. Initially announced in November 2024, the deal closed on January 13, 2025, marking a significant step in Red Hat’s strategy to enhance its AI capabilities across hybrid cloud environments.

While the price was not disclosed, Neural Magic had previously raised $50 million in venture capital funding from various investors, including Andreessen Horowitz, New Enterprise Associates, Amdocs, and Comcast Ventures. We can safely assume that the deal wasn’t made for peanuts. The cause probably wasn’t hurt, I’m sure, by the fact that Neural Magic’s CEO before the deal closed was Brian Stevens, Red Hat’s former CTO and executive VP. 

Also: The best open-source AI models: All your free-to-use options explained

When the merger was first announced, Stevens said: “At Neural Magic, our vision is that the future of AI is open, and we have been on a mission to enable enterprises to capture the powerful innovation from AI while at the same time being free of friction and restriction. Open-source models, optimization algorithms, and inferencing systems are the heart of this — enabling enterprises to own their own AI models, privately customize them to their datasets, and deploy them on their private multi-vendor infrastructure — cloud, data center, or edge.”

Neural Magic, an MIT spinoff founded in 2018, has developed innovative software and algorithms that accelerate generative AI inference workloads. 

The company’s technology allows complex AI models to run efficiently on commodity hardware, including standard CPUs and GPUs, potentially reducing the need for expensive, specialized AI accelerators. 

Also: Nearly half of Gen AI adopters want it open source – here’s why

So, what does that capability mean? Well, Neural Magic brought several key technologies to Red Hat through the acquisition:

1. vLLM Expertise: Neural Magic is a leading contributor to vLLM, an open-source project for efficient large language model (LLM) serving. In particular, vLLM is already Red Hat Enterprise Linux (RHEL) AI and Red Hat OpenShift AI‘s inference engine. This move clearly benefits Red Hat’s existing AI programs. 
2. DeepSparse: An inference runtime that delivers GPU-class performance on commodity CPUs through algorithms that reduce computation and memory needs for neural network execution. Specifically, besides LLMs, DeepSparse is also useful for computer vision (CV) and natural language processing (NLP).
3. SparseZoo: If you use DeepSparse, Neural Magic already provides a repository of pre-optimized CV, NLP, and LLM models. 
4. Model Compression Techniques: Neural Magic has developed advanced methods for model quantization and sparsification, which can significantly reduce model size and computational requirements without substantial loss in accuracy.
5. Cross-platform Optimization: Neural Magic’s technology enables AI model optimization for deployment across cloud, data center, and edge environments.

These technologies align with Red Hat’s vision of making AI workloads more accessible, efficient, and deployable across hybrid cloud environments, potentially reducing the need for specialized and expensive AI hardware. In a nutshell, Neural Magic’s technology enables its users to do AI on the cheap. It’s also, at its vLLM heart, an open-source AI approach. 

Also: Wanted: Humans to build robots for OpenAI – and not everyone is thrilled

For Red Hat, as the company’s CEO Matt Hicks said in a statement: “We’re thrilled to complement our hybrid cloud-focused AI portfolio with Neural Magic’s groundbreaking AI innovation, furthering our drive to not only be the ‘Red Hat’ of open source, but the ‘Red Hat’ of AI as well.” 

On X, Hicks wrote: “At @RedHat, we believe the future of AI is open. That’s why I’m incredibly excited about our acquisition of @NeuralMagic. Together, we’re furthering our commitment to our customers and the open-source community to deliver on the future of AI — and that starts today.”

Strategically, Red Hat hopes its Neural Magic acquisition will bolster Red Hat’s position in its ability to deploy AI workloads across diverse computing environments, from on-premises data centers to public clouds, and to improve the efficiency of AI workloads already running on RHEL AI, OpenShift AI, and InstructLab, an open-source Red Hat and IBM AI community project to improve open source-licensed IBM Granite LLMs.

Also: 15 ways AI saved me time at work in 2024 – and how I plan to use it in 2025

So, will this approach work out? Dave McCarthy, IDC‘s Cloud and Edge Services research vice president, thinks so: “Red Hat’s acquisition of Neural Magic is a strategic enhancement to its AI capabilities, facilitating AI deployment across hybrid clouds by leveraging Neural Magic’s expertise in model optimization and inference acceleration. This move aligns with Red Hat’s commitment to open-source innovation and positions the company to offer more cost-effective, scalable AI solutions that reduce dependency on specialized hardware.”