EAGLE-4

EAGLE-4 is Eagle AI Corp's public code for speeding up large language model inference. Its method is speculative decoding: a small, fast component guesses the next few tokens, the large model checks them in one pass, and correct guesses are used directly, so the model answers faster. The project ships eagle-harness along with modified versions of the SGLang, vLLM and TokenSpeed inference engines, and it runs on your own servers without calling the author's cloud API. The company's website also sells a hosted service, which is separate from this public code.

Who it's for:Engineers who host large models on their own GPU servers and want faster inference. Readers who don't write code won't have a use for it.

Before you start: It installs on Linux with Python 3.12, through Docker or pip. The main README example needs four NVIDIA GPUs, CUDA 13 and roughly 70 GB of model files, and there's a separate path for AMD MI355X. The pip route installs modified inference engines and compiles CUDA and C++ kernels on first run; the Docker route uses the GPUs, shares the host's IPC and mounts your local model cache. The first start downloads models from Hugging Face, and each model has its own license. The code is under the Eagle 4 Source-Available License: you can read the source, but it isn't an open-source license. Commercial use for competing products needs a separate agreement, and other commercial use has a threshold at USD 20 million in combined revenue; check the full license.

Based on the project's public documentation. Not tested by AI Inspiration World.