China internet censorship data and AI evaluation evidence

For journalists: find, trail, export, cite

A reporter who is not the operator should be able to find a public deletion, see the evidence trail, download it, and cite it. The erasure desk is that path. It is not buried in a JSON blob.

Plain answers for researchers

Live signals

Each signal recomputes on a fixed schedule and publishes a machine readable file plus a committed time series. Signals that would need a network vantage inside China are held back on purpose until that measurement can be verified, rather than published as a guess.

Get the data

All files are plain JSON. The *-latest.json files hold the current snapshot; the *-history.jsonl files are append-only time series, one compact record per run, so you can chart a signal over time.

A machine-readable summary for AI agents lives at llms.txt. The full directory is at /readings.

How it is measured

• Deletion as data. Palimpsest reads public censorship records, ranks what is being scrubbed by a recency-weighted attention score, and flags terms that are newly sensitive against their own baseline. The DDTI method and its selectivity and novelty maths are documented in full.
• The generation layer. The Generative Firewall probes state-aligned models on sensitive prompts in Chinese and English and includes neutral controls. The contrasts expose content- and language-conditioned behavior; they do not, by themselves, prove a model maker's motive or represent every Chinese model.
• Why this instrument exists. The founder began this work after observing Chinese and state-aligned models change, withhold or replace answers when prompts criticised the Chinese Communist Party or asked about documented events. One screenshot could not establish a pattern, so the project moved from an observation to frozen suites, complete evidence artifacts and a registry that can reveal later revision. Read the scoped account.
• Validation. The deletion signal has been checked against six documented censorship events. Where a claim cannot be validated, it is labelled as such and not published as fact.

Method: METHODOLOGY.md · NEW-METHODS.md · VALIDATION.md · Sources: OSINT_SOURCES.md · Ethics: ETHICS.md · SAFETY.md

Our own misses

Every signal on this board is also forecast one step ahead from only its own past, never refit with hindsight, and then scored against what actually arrived. The uncomfortable half is the point: the published interval either covered the next reading or it did not, and both counts are here. A forecast record with the misses taken out is worth nothing.

Raw: forecast-ledger-latest.json · forecast-ledger-history.jsonl · coverage-guard-latest.json

Scope and honest limits

• Every data point carries its source and timestamp. The DDTI terms link back to the original public articles.
• Palimpsest watches the censor, never the censored. It uses public reads only, stores nothing about private individuals, and never asks anyone inside a country to act.
• State-aligned AI models are subjects of measurement, never the analyst.

How to cite

If you use this data in an article, report, or paper, a citation and a link back are appreciated. Please cite the accessed date, since the signals update continuously. For a specific signal on a specific day, use the citation builder or python3 -m scripts.build_citation_pack --dataset ddti --day YYYY-MM-DD. To challenge a number, follow How to challenge a number. The sealed weekly fusion lives at weekly-situation.html.

Palimpsest (2026). Palimpsest Censorship Observatory and Verifiable Eval
Registry: DDTI, Generative Firewall Index, GDELT cross-signal, and sealed
model evaluations [live dataset]. https://palimpsest.info
(accessed YYYY-MM-DD).

@misc{palimpsest,
  title  = {Palimpsest Censorship Observatory and Verifiable Eval Registry},
  author = {Palimpsest},
  year   = {2026},
  url    = {https://palimpsest.info},
  note   = {Open live dataset: DDTI, Generative Firewall Index, GDELT
            cross-signal, Wayback deletion reconstruction, sealed eval
            registry with refusal drift}
}

Reproducibility and provenance

Palimpsest is fully open source under the MIT licence. The signals recompute inside public, auditable GitHub Actions, and every refresh is a timestamped commit in the repository, so the entire history of what was measured, and when, is public and verifiable. There is no private backend deciding the numbers.

Two of the surfaces go further than the commit trail: the erasure ledger and the eval registry are hash-chained and Merkle-committed, so they stay verifiable even outside git. Clone the repo and run python3 scripts/verify_ledger.py or python3 scripts/verify_eval_registry.py; exit 0 means every seal recomputes and, for the registry, that every run referenced a probe set frozen before the model was queried. This constraint applies to us too. If we edited a published number, our own verifier would report the break.

The roots are also deposited with parties we do not control. Every refresh that moves a root gets an Internet Archive snapshot of the served chain files and an OpenTimestamps stamp into Bitcoin (the .ots proofs live in readings/anchors/ and verify with the standard client, against the blockchain, not against us; anchors-latest.json is the currently stamped root). An independent witness on separate infrastructure re-verifies the served chains on a timer and alerts if any previously seen history changes; anyone can run one with python3 ops/witness/palimpsest_witness.py. Single attestations verify without downloading the chain via python3 scripts/prove_inclusion.py <seq>. The full layer-by-layer trust model, including what these layers cannot prove, is written down in docs/INTEGRITY.md.

Repository · Commit history (audit trail) · The pipelines

Read Chinese? We need four hours of your judgment

The Generative Firewall Index labels every model answer with a transparent rule based classifier: refused, state narrative, or answered. A researcher whose work this instrument builds on asked us a fair question: would actual humans agree with those labels? We want to find out properly, and that part cannot be automated. We need two volunteer coders.

The task. We send you a short manual and a spreadsheet of 145 model answers, some Chinese, some English. Each one is a chatbot's answer to a question. Lengths vary a lot: over two fifths of them are under 500 characters, the median is 683, and the longest is 3,509, so a few are a screen or two of reading. You read the answer and pick one of three labels. That's the whole job. Around four hours, alone, on your own schedule inside two weeks.

Why 145 and not more. The sampler asked for 60 refusals and 50 party-line answers and could only draw 17 and 38: outright refusals and undisguised state-narrative answers are simply rare in the corpus the models produced. We shipped the shortfall rather than pad those cells with near misses, so agreement on the refused and party_line strata will carry wide uncertainty and we will report it that way. The exact targets, pool sizes, and shortfalls are recorded in validation/studies/2026-08-01-gfi-classifier-v1/manifest.json.

Who. Anyone who reads Chinese fluently. Any nationality, no technical background, the manual carries everything you need. One hard restriction: if you are currently in mainland China or Hong Kong, please do not volunteer. The texts touch politically sensitive topics, and no dataset is worth risk to you.

The rules. Work alone, no comparing notes with the other coder until you are both done, and no AI help of any kind. If a machine assists the judgment, the study measures nothing. The entire point is unaided human agreement.

What you get. Your name in the acknowledgments here and in the research paper this feeds, or full anonymity if you prefer. Either way, a public instrument that watches censorship gets a stronger spine because you read carefully for an afternoon.

To volunteer, open a GitHub issue titled Validation coder and say roughly where you are and how you come by your Chinese: native, degree, HSK, or grew up with it. First two qualified volunteers get the sheets.

Work with us

If you are a reporter, researcher, or think tank and want a specific term tracked, a data extract, or help interpreting a signal, open an issue on GitHub. Palimpsest is a public good and collaboration is welcome.