Discover great AI tools
📢 NoticeWebsite launched:DeepToSeek AI tools directory is live. Submit great tools and get featured!
Curated on GitHub

Open Source Radar: Inference & Training

Scanning GitHub for open-source AI projects that are actually worth running: self-hostable, forkable, with a clear license. Star counts and languages are verified against each repository.

148Projects
8.8mTotal stars
10Categories
5 open-source projects
promptfoo promptfoo/promptfoo

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

cici-cd
27.4k TypeScript MITUpdated 2d ago
opik comet-ml/opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

evaluationlangchain
23.4k Java Apache-2.0Updated 2d ago
OpenLLM bentoml/OpenLLM

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

bentomlfine-tuning
12.6k Python Apache-2.0Updated 3d ago
L
litellm BerriAI/litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

ai-gatewayanthropic
63.4k Python MIT
V
vllm vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

cudainference
97.1k Python Apache-2.0