Neural Magic: Models, Compression techniques, Libraries, and Languages.

What is Neural Magic

Neural Magic offers a range of neural network / LLM optimization products. Their statement is "LLMs on CPUs. Period."

This, while being a bold statement, is big for companies that rely on LLMs, as running an unoptimized LLM can become a costly feat. Neural Magic has both Post Training and Training Aware products.

Neural Magic supports YAML configuration files, called Sparsification Recipes to configure how the compression should be applied, at which levels and how intense.

The configuration can be automatically generated too, so you'll have a place to start finetuning.

The company is constantly pushing updates, as we see regular activity on the repositories on github. All in all, a feature rich, and expert library to optimize your neural network. Neural Magic does not only compress LLMs even though it seems to be the main focus for the company at the moment.

The executive team contains a bunch of highly skilled and higher-ups from large corporates like Google, Amazon, MIT. The list of Investors is as impressive, containing Andreessen Horrowitz and VMWare to name a few.