Matrix-Game 4.0

Matrix-Game 4.0 is an open-source research project from Skywork AI's Matrix-Game team, which describes it as "Streaming neural rendering for executable low-poly games." A conventional game engine handles gameplay, physics and input, while a causal video model renders the picture live from the engine's low-poly frames; the HUD and sound are layered on separately, outside the model. This release includes three scenes: two shooters set in a medieval castle and ancient Egypt, and a city racing scene with time-trial and pursuit modes. On a single A100 80 GB, the team measured about 38 to 39 frames per second at 864×480; those are server-side figures, not the frame rate or latency you'd see in a browser. The player's browser needs no GPU, since the picture streams from the machine running the model.

Who it's for:game, graphics and AI video teams and researchers, and technical decision-makers who want to see for themselves how far live AI rendering of games has come. Without a high-end GPU and a Linux server, you can only watch the demos on the project page for now.

Before you start: running the model takes Linux, an NVIDIA GPU and Python 3.12. The reference setup is an A100 80 GB; there's also a build for the RTX 5090, and the team says RTX 5080 and 5070 support is still on the way. Installation pulls in PyTorch, CMake, SDL2, FFmpeg, MediaMTX and more, plus model weights and scene assets from Hugging Face, so set aside plenty of disk space. The team says it's a demo for a single user on a trusted local machine, and its ports shouldn't be exposed directly to the internet. The project code is Apache-2.0; the models and game assets carry their own licenses, so check each before use.

Based on the project's public documentation. Not tested by AI Inspiration World.