Overview
Neural Magic built software to make trained machine learning models cheaper or faster to run. Its early work focused on running optimized models on ordinary CPUs. Later, it contributed model compression and performance work to vLLM, an open-source system for serving large language models on accelerators.[1][2]
Red Hat completed its acquisition of Neural Magic in January 2025. Later that year, the team deprecated its earlier DeepSparse product family and shifted toward vLLM-based work.[3][4]
Founding Story
MIT professor Nir Shavit and research scientist Alexander Matveev founded Neural Magic in 2018. They had worked on algorithms for running neural networks on large biological datasets using standard computers. Their company set out to make AI inference—the process of applying a trained model to new inputs—less dependent on specialized hardware.[5][6]
The hardware claim was ambitious. The company said its software could deliver GPU-class performance on commodity CPUs for certain models. Such comparisons depend on the model, hardware, accuracy, and workload; the funding announcement does not establish that every CPU deployment would match a GPU.[1]
Timeline
- 2018: Shavit and Matveev found the company from their MIT research.[5]
- 2019: Neural Magic announces $15 million in seed funding and early access to an inference engine for CPUs. Andreessen Horowitz participates.[6]
- 2021: The company announces a $30 million Series A, bringing reported funding to $50 million; a16z participates again.[1]
- 2024: Neural Magic describes its work on model compression and vLLM, including GPU inference optimizations.[2]
- 2025: Red Hat completes its acquisition in January. DeepSparse and related community tools are deprecated in June.[3][4]
What They Built
Neural Magic's DeepSparse was an inference runtime for CPUs. It could execute models prepared with its associated tools. SparseML helped prepare models, while SparseZoo distributed ready-made models. A developer could select or prepare a model, export it in a supported format, and run it through DeepSparse for tasks such as image classification.[4][7]
One optimization is pruning: removing model parameters that contribute little to its output. The resulting sparse model has many zero values. Another is quantization, which represents values with fewer bits. Both can reduce the data that must be stored or moved, but may change accuracy. The runtime must exploit that representation for the reduction to translate into speed.[7][8]
As language models became more prominent, the team also worked on efficient GPU inference. Its 2024 description of LLM Compressor explains how developers could compress models for deployment with vLLM. This was an expansion of the efficiency problem, not evidence that its original CPU runtime became a GPU service.[2]
Market Position
Target Customers
The intended users were data scientists and engineers deploying trained models. An organization with existing CPU capacity could evaluate whether a compressed model met its speed and accuracy requirements without purchasing additional specialized hardware. The appropriate choice still depended on its workload.[6]
Market Size
The sources reviewed do not establish a reliable market size for Neural Magic's particular software or a count of paying customers.
Competition
The company competed with specialized hardware and other optimization software, including offerings from larger chip companies and startups. Today vLLM is an open-source inference engine, and Red Hat sells an inference product built around it. A new entrant must test its value against that active ecosystem.[9][10]
Business Model
The company distributed developer tools and models, including community versions and a DeepSparse Enterprise offering. The sources available here do not establish a consistent price, contract size, or revenue breakdown. The sources do not disclose how many users became paying customers.[4]
Traction
Neural Magic reported $50 million raised after its 2021 Series A. Its repositories and contributions show software released for developers, but the reviewed sources do not establish revenue, profitability, or customer retention.[1][2]
Outcome and evolution
Efficiency work moved into Red Hat
Red Hat said it acquired Neural Magic for its inference engineering and model optimization expertise. The deal closed on January 13, 2025; its price was not disclosed. The announcement supports the strategic fit, but does not establish the startup's financial results or each asset's contribution to the deal.[3][9]
The team announced that DeepSparse, SparseML, SparseZoo, and Sparsify community versions were deprecated on June 2, 2025 and would receive no further updates or support. Red Hat's current inference platform uses vLLM and includes model optimization tooling. Users of the older products must therefore distinguish available source code from an actively supported service.[4][11]
Key Lessons
- Measure the complete workload. Smaller models help only when the runtime and hardware can use their representation without unacceptable accuracy loss.
- Follow the bottleneck. Neural Magic applied its efficiency research to CPU inference, then also to language models on GPUs.
- Check support after acquisition. Its original community tool family was deprecated even as related expertise continued at Red Hat.
- Treat funding and usage as partial evidence. Neither establishes the economics of the business.
Sources
- Neural Magic, 2021 financing
- Neural Magic, LLM Compressor
- Red Hat, completed acquisition
- DeepSparse repository, end-of-life notice
- Neural Magic, court complaint and founding history
- TechCrunch, 2019 seed round
- DeepSparse image classification documentation
- Red Hat Research, model optimization interview
- TechCrunch, acquisition announcement
- Red Hat, AI Inference
- Red Hat, supported configurations