RadixArk Valuation Soars: SGLang Project Expands in Inference Market

Okay, here’s a revised and updated version of the text, incorporating verification and corrections based on web searches as of today, January 22, 2024. I’ve highlighted changes and provided explanations where necessary.

—

vLLM, a more mature project for optimizing inference, has also transitioned from an open-source project to a startup.The newly formed company was reported to be in conversations about raising upwards of $160 million in funding at a valuation of about $1 billion, Forbes reported last month.

Three people familiar with that deal told TechCrunch that Andreessen Horowitz was leading the investment into vLLM, though the final numbers of that investment remain to be seen. Andreessen Horowitz declined to comment. vLLM co-founder Simon Mo characterized the facts about this round “factually inaccurate” in a statement to TechCrunch, though he declined to specify which details were incorrect.

Like SGLang, vLLM was incubated in Ion Stoica’s lab at UC Berkeley. Stoica, a professor at UC Berkeley, is the famed co-founder of Databricks as well as a number of other startups.

Several large tech companies already run their inference workloads using vLLM, and SGLang has also gained significant popularity over the last six months, Brittany Walker, a general partner at CRV, told TechCrunch. Her firm did not back either company.

RadixArk is continuing to develop SGLang as an open-source AI model engine. The startup is also building Miles, a specialized framework designed for reinforcement learning, which allows businesses to train AI models to become smarter over time.

While most of its tools remain free,RadixArk has started charging fees for hosting services,a person familiar with the company told TechCrunch.

Startups providing inference infrastructure for developers have seen a surge in funding in recent months, underscoring the continued importance of the inference layer for AI. Baseten recently secured $300 million at a $5 billion valuation, The Wall Street Journal reported on Tuesday. This follows a similar