Discover great AI tools
📢 NoticeWebsite launched:DeepToSeek AI tools directory is live. Submit great tools and get featured!
Curated on GitHub

Open Source Radar: Inference & Training

Scanning GitHub for open-source AI projects that are actually worth running: self-hostable, forkable, with a clear license. Star counts and languages are verified against each repository.

148Projects
8.8mTotal stars
10Categories
21 open-source projects
pytorch pytorch/pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

autograddeep-learning
105.8k Python Updated 2d ago
mlflow mlflow/mlflow

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

agentopsagents
28.9k Python Apache-2.0Updated 2d ago
Megatron-LM NVIDIA/Megatron-LM

Ongoing research training transformer models at scale

large-language-modelsmodel-para
18.4k Python Updated 2d ago
langfuse langfuse/langfuse

🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.

ai-gatewayanalytics
37.6k TypeScript Updated 2d ago
ray ray-project/ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

data-sciencedeep-learning
44.3k Python Apache-2.0Updated 2d ago
phoenix Arize-ai/phoenix

AI Observability & Evaluation

agentsai-monitoring
12.3k Python Updated 2d ago
promptfoo promptfoo/promptfoo

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

cici-cd
27.4k TypeScript MITUpdated 2d ago
DeepSpeed deepspeedai/DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

billion-parameterscompression
43.3k Python Apache-2.0Updated 2d ago
axolotl axolotl-ai-cloud/axolotl

Go ahead and axolotl questions

fine-tuningllm
12.7k Python Apache-2.0Updated 2d ago
opik comet-ml/opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

evaluationlangchain
23.4k Java Apache-2.0Updated 2d ago
openvino openvinotoolkit/openvino

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

aicomputer-vision
11.2k C++ Apache-2.0Updated 2d ago
airllm lyogavin/airllm

AirLLM 2.8T LLM inference with single 4GB GPU

chinese-llmchinese-nlp
42.6k Jupyter Notebook Apache-2.0Updated 2d ago
transformers.js huggingface/transformers.js

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!

browserjavascript
16.4k JavaScript Apache-2.0Updated 2d ago
BentoML bentoml/BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

ai-inferencedeep-learning
8.9k Python Apache-2.0Updated 3d ago
OpenLLM bentoml/OpenLLM

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

bentomlfine-tuning
12.6k Python Apache-2.0Updated 3d ago
litgpt Lightning-AI/litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

aiartificial-intelligence
13.8k Python Apache-2.0Updated 9d ago
llamafile mozilla-ai/llamafile

Distribute and run LLMs with a single file.

cross-platformgguf
26.7k C++ Updated 11d ago
L
litellm BerriAI/litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

ai-gatewayanthropic
63.4k Python MIT
U
unsloth unslothai/unsloth

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.

fine-tuningllm
79.7k Python Apache-2.0
T
transformers huggingface/transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

audiodeep-learning
169.2k Python Apache-2.0
V
vllm vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

cudainference
97.1k Python Apache-2.0