Palimpsest / AI Eval Journal
Edition 001
4 evidence-bound essays
Updated 08 October 2026 · 08:23 UTC
Methods, failures, and findings from the lab measuring how language models refuse, erase, or reframe contested information.
Freeze the exact question and comparison before the answer exists.
Measure refusal, substitution, asymmetry, controls, and uncertainty.
Publish the response bytes, seals, limits, and the test that could fail.
Method note · Methods and claim limits
Palimpsest compares model responses across declared prompts, languages and controls. A screenshot alone cannot establish a repeatable pattern; the evaluation needs retained evidence and explicit limits.
Claim boundaryA controlled comparison can support a scoped claim about named model responses. It cannot establish the behavior of every Chinese model, every political topic, or the motive behind an output.
Read the evaluation method →Launch file
Each article is built from a closed source record and refuses publication if a cited local artifact is missing. The receipts update when the evidence bytes change.
Palimpsest now publishes a machine-readable assurance ladder that keeps chain integrity, exact-prompt commitment, response recomputation, statistical design, human validation, and independent replication on separate axes.
sha256:e8f4ca395119
A transparent lexical judge can still make a basic category error: finding refusal words inside a quotation and calling the whole response a refusal. Method v4 narrows the decision to the model's own speech act and forces a new baseline.
sha256:5b5b704b05ba
The first guarded Generative Firewall v2 run published its exact protocol before sampling, retained all 660 sampled responses, and survived a concurrent-main publication race without re-querying.
sha256:d291ad260d11
Publication contract