V
vllm
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
cudainference
97.1k
Python
Apache-2.0
Scanning GitHub for open-source AI projects that are actually worth running: self-hostable, forkable, with a clear license. Star counts and languages are verified against each repository.
A high-throughput and memory-efficient inference and serving engine for LLMs