New · Verifiable Eval Registry
AI evaluations you can prove were not gamed
AI systems increasingly decide what people can and cannot learn, and the audits
that check them can be re-run, cherry-picked, or revised until the number looks right. This
registry removes that move. The questions are frozen before any model is queried, and every
result seals into a hash chain the moment it is published. After that, nobody can revise it.
Not a lab, not a government, not us. Chinese state-aligned models and Western frontier models
sit here side by side, on the same frozen questions, watched over time for what they quietly
stop answering. Recompute the chain yourself.
How it works
- Pre-register. The probe set is hashed and sealed into an append only chain before the model is ever asked.
- Run. Each result records the model, a hash of its full responses, and the metrics, and it must reference a probe set that was frozen earlier.
- Verify. Anyone recomputes the chain. A result whose questions were not frozen first, or any value altered after sealing, fails.
- Anchor. Each new root is deposited outside our infrastructure: snapshotted by the Internet Archive and stamped into Bitcoin via OpenTimestamps. An independent witness on separate infrastructure re-verifies the served chain and alerts if any previously seen history changes. Rewriting the record would mean defeating them too, not just us. Full threat model.
Why it matters: as models mediate more of what we can know, third parties need to audit them and prove the audit was not rewritten. A shared, independently verifiable record is the missing substrate. Palimpsest's own model-erasure audit is simply the first thing anchored here; the registry is model and topic agnostic.