MiniMind

MiniMind is an open-source project by GitHub user jingyaogong. Its README says the aim is to train an ultra-small language model of about 64M parameters completely from scratch, and that it is also a tutorial for getting started with LLMs. The project gives the full model architecture and training pipeline, covering pretraining, supervised fine-tuning (SFT), LoRA, reinforcement learning, tool use and distillation, and it supports a mixture-of-experts (MoE) structure. The core algorithms are written step by step in plain PyTorch instead of third-party wrappers, so you can follow every step. The README is in Chinese and English.

Who it's for:developers with Python and PyTorch experience and a GPU who want to take a small language model from pretraining through fine-tuning and deployment themselves. Readers who don't write code won't get much from it.

Before you start: the repository is free and the code is under the Apache-2.0 licence. Training needs your own GPU or a rented cloud GPU. The author notes that "2 hours" is the time for one epoch of the SFT stage on a single NVIDIA 3090, and "3 yuan" is the GPU rental for that time, not the cost of the whole training process. If you turn on external training logs such as wandb or swanlab, or use hosted inference or cloud GPUs, your training records and data may be uploaded to those services. Check the licences of training data and third-party models separately.

Based on the project's public documentation. Not tested by AI Inspiration World.