Magpie Intelligence Check

Magpie Intelligence Check is a plugin for the Magpie gateway by 木子不是木子狸; the README describes it as a quality probe for your chosen main model. You add a model's full ID to monitoring and the plugin tests it with a fixed reasoning question. If the model hasn't been tested yet, or the last result is more than 15 minutes old, the request is blocked; once the background test finishes, sending it again goes through. If the model gets it wrong, the plugin keeps holding requests until you choose to re-test, allow traffic for 15 minutes, pause or stop monitoring. Six more questions only save the answers for you to judge and don't affect the gate. The author notes that failing one question doesn't prove a model was swapped, and that this isn't an identity check or a ranking.

Who it's for:Developers who use one main model through a Magpie gateway over time and want early warning of a drop in quality. Readers who don't write code won't get much from it.

Before you start: You need a running Magpie gateway and its key. You can install by entering github:mu-zi-lee/magpie-intelligence-check in Magpie's plugin box, which runs a precompiled Go background program from the repository, or use a release package or build with Go 1.26. Test questions go through the gateway to the monitored model and use real calls and tokens; by default each model gets up to 120 probe requests a day, which you can change. The gateway key is stored in a local database file. The plugin can open the local admin page to the internet through a temporary Cloudflare tunnel; we don't recommend turning that on, since the local admin page is enough. MIT-licensed.

Based on the project's public documentation. Not tested by AI Inspiration World.