Ongoing research training transformer models at scale
About this project
Megatron-LM is an open-source inference and training project on GitHub: Ongoing research training transformer models at scale. It is infrastructure rather than a finished product: the layer that serves, quantises, trains or orchestrates models. It is maintained by NVIDIA and published as a public repository you can read, fork and run yourself.
On GitHub the repository is tagged large-language-models, model-para and transformers, which is a fair summary of what it covers. That generally means an API surface, batching and memory management, and often an OpenAI-compatible endpoint so existing clients keep working unchanged.
It is written mainly in Python and released under no standard SPDX license identifier — the repository uses a custom or modified license, so read it before commercial use. It is actively maintained: the most recent commit landed on November 18, 2026.
To try it, clone the repository and follow its README; that file is the canonical install guide. There is also a project page at https://docs.nvidia.com/megatron-core/developer-guide/latest/get-started/quickstart.html for documentation or a hosted demo. Because it runs on your own infrastructure, budget time for environment setup: dependencies, model weights and configuration usually take longer than the first successful run.
Key facts as recorded on GitHub: 18,434 stars; 4,749 forks; primary language Python; last commit on November 18, 2026; topics large-language-models, model-para, transformers.