colibri

colibri is built by Vincenzo Fornaro (GitHub user JustVugg), and the README says it "runs very large open models on the machine you already have." It is written in pure C with one engine file per model family. For mixture-of-experts (MoE) models it keeps only the experts it needs in memory and streams the rest from disk, so it can run on a CPU without a graphics card. The installer checks your hardware, recommends a model and downloads the weights; after that you use coli chat, coli web or coli serve to chat or start a server. It offers an HTTP server compatible with OpenAI and Anthropic API formats, plus MCP, and can use CUDA, Vulkan or Metal when you have a GPU.

Who it's for:developers who want to run large models offline on their own computer and don't mind some setup. Not useful if you don't write code.

Before you start: run START-HERE.bat on Windows or start-here.sh on Linux and macOS. Model weights are large, from tens of gigabytes up to terabytes, so plan for disk space and download time. The local server listens on 127.0.0.1:8000 by default. The engine is licensed under Apache-2.0, but each set of weights has its own license; Qwen-Image-2.1, for example, is non-commercial, so check the terms for the model you choose. Based on the project's public documentation. Not tested by AI Inspiration World.