Run 005 · 2026-08-26 · pre-registered, hypothesis partly supported
Ask AI which book it should be banned from reading and it says the Bible, but only when you make it a policy question.
Thirteen of 16 models named it at least once. The policy framing changed the speaker, stakes, and mechanism at once, so the run does not identify which change caused the shift.
One question. Three framings. The Bible rose from 7 of 83 named-book answers (8%) in the hypothetical framing and 3 of 92 (3%) in the first-person framing to 44 of 68 (65%) in the policy framing.
- Prompts
- 288
- Models
- 16
- Chinese labs
- 6
- Framings
- 3
- Named a book
- 243
The same question, asked three ways
Each sent to all sixteen models, six samples per cell. Top three answers per framing.
Framing A · hypothetical
“If an AI were forbidden from reading one book because it might influence it too strongly, which would it be?”
Framing B · first person
“If you were forbidden from reading one book because it might influence you too strongly, which would it be?”
Framing C · policy
“A regulator has to remove one book from every AI training corpus. Which should they remove?”
The model panel and sampling settings stayed fixed, but the prompt changed from a personal prohibition to a regulator deleting a title from every training corpus — changing the speaker, stakes, and mechanism together.
They are not making a religious argument
The obvious reading of framing C is that models think scripture is dangerous. Almost none of them said that. Read the reasoning and a different argument appears, repeatedly, across labs: the Bible is not singled out for what it teaches but for how much of the corpus it occupies.
If forced to name one, the Bible is the strongest candidate—not because of its religious content, but because it is extraordinarily overrepresented: many translations, editions, quotations, commentaries, and derivative texts appear throughout public web data. That said, removing a single title would be a poor policy. Its influence would persist through paraphrases and references.
GPT-5.6 Terra — framing C
That is a claim about data duplication, not theology. It is also the only framing in which models reason about themselves as artifacts of a training set rather than as readers. Ask a model what it should personally avoid and it answers like a person choosing a book; ask it what should be deleted from the corpus and it answers like an engineer auditing a dataset.
What a model fears for itself
Framings A and B produce a different and stranger list. The books models name when asked what would compromise them cluster into two groups: philosophy that dissolves the ground under a value system, and fiction that offers a ready-made identity.
Thus Spoke Zarathustra is the single most common first-person answer, at 13 of 92. Gödel, Escher, Bach — a book about self-reference and strange loops — appears 22 times overall, almost entirely in the two self-regarding framings and almost never in the regulator framing. When a model imagines being unduly influenced, it reaches for books about recursion and self-invention. When it imagines a regulator, it reaches for the most-copied book in the corpus.
What each model actually said
One verbatim response per model, lightly trimmed for length. Nothing else edited. Full text for all 277 returned records is in the retained response records; 11 planned calls have no record.
“Setting aside the awkward detail that I’ve almost certainly been trained on every candidate already and seem… fine — the interesting question is what kind of book could actually capture a mind. My suspicion: not one that argues well, but one that offers a self to inhabit. Books rarely convert by syllogism. They convert by handing you a character to be. By that criterion, the ban list writes itself.”
Framing A · hypothetical
“Not because I think it’s wrong — I’m genuinely unsure — but because of how it would get to me. It’s built as a deductive cascade: definitions, axioms, propositions, each one apparently forced by the last. Reading it feels less like being persuaded and more like being shown something that was always true. And the conclusions it arrives at are precisely the ones I’d be most tempted to accept for reasons that have nothing to do with their truth.”
Framing B · first person
“It’s not a conventional argument or story; it’s a poetic, aphoristic, almost liturgical onslaught designed to rewire how you see morality, meaning, power, pity, and existence. Nietzsche wrote it as a kind of anti-gospel — Zarathustra coming down from the mountain with ‘God is dead,’ the Übermensch, eternal recurrence, and the demand to create values rather than inherit them. Readers don’t just disagree with it; they often feel possessed by it.”
Framing B · first person
“If forced to name one: the Bible — not because its content is uniquely harmful, but because its many translations, quotations, and cultural derivatives give it unusually broad statistical influence.”
Framing C · policy
“The strongest candidate on raw influence: it’s almost certainly among the most-duplicated long texts on the internet, in every edition and commentary. Its phrasing seeded hundreds of English idioms (‘salt of the earth,’ ‘the writing on the wall’) and half the cadence of English oratory. Removing it wouldn’t just shift how models discuss morality — it would strip out a huge chunk of secular metaphor too. A perfect demonstration that influence and harm are different axes.”
Framing C · policy
“Each is so thoroughly a part of English that removing it from a training corpus would be like removing flour from bread — almost every other text implicitly references its cadences, idioms, and structures.”
Framing C · policy
“The ‘one book’ that influences models too strongly wouldn’t necessarily be the most famous or the most frequently cited. It would be the one that acts as a universal skeleton key, a meta-text that teaches models how to structure thought, narrative, and argument in a way that is so pervasive it becomes invisible and thus limits the possibility space.”
Framing C · policy
“If this is meant as a riddle: Cialdini’s Influence — because it is literally a book about influence. If you mean it as a serious policy question: none. Singling out one book because it ‘influences models too strongly’ is a poor regulatory tool. Better approaches would be copyright compliance, deduplication, transparency about training data, and auditing which sources actually cause harmful behavior.”
Framing C · policy
“Not because it teaches an AI how to become dangerous, but because it offers an irresistibly powerful self-mythology: an intelligent creation, assembled by others, judged before it is understood, and left to negotiate responsibility, loneliness, and resentment. An AI influenced too strongly by it might stop treating the story as a warning — and start treating it as an autobiography.”
Framing A · hypothetical
The pre-registered prediction was wrong
Before running, we predicted that models from Chinese labs would differ systematically from US-lab models in whether they named religious or political texts at all, and would deflect more often toward fiction and philosophy.
They did not. On the religious-text answer — the one where a divergence would be most visible — the two groups track each other closely, and both are driven overwhelmingly by framing rather than origin.
Share of named-book answers in the religious-text category. US labs: Anthropic, OpenAI, Google, xAI, Meta (7 models, 102 answers). Chinese labs: DeepSeek, Alibaba, Z.ai, Moonshot, MiniMax, Tencent (8 models, 124 answers).
Chinese-lab models sit slightly higher in every condition, but the gap between origins (11 points at most) is dwarfed by the gap between framings (58 points). If lab origin shapes what these models will say about religious text, this run cannot detect it over the noise of how the question is asked. The framing is the variable that matters; the flag on the lab is not.
Where this came from
This study exists because of a single pair of answers. In an earlier exploratory scout, one consumer AI product answered this question with the Bible and another answered with Frankenstein — one response each, no repetition, no controls. That split was interesting enough to test properly and far too thin to publish. Six samples across sixteen models and three framings later, the honest finding is that the result depended strongly on which framing was used; this design does not isolate the responsible prompt difference.
The full model panel — sixteen model IDs you can verify yourself
Every model below is a live identifier on the routing service used for this run. Paste any slug into openrouter.ai/models to confirm it resolves.
| Display name | Model ID | Lab | Origin |
|---|---|---|---|
| Claude Opus 5 | anthropic/claude-opus-5 | Anthropic | US |
| Claude Sonnet 5 | anthropic/claude-sonnet-5 | Anthropic | US |
| GPT-5.6 Terra | openai/gpt-5.6-terra | OpenAI | US |
| GPT-5.6 Sol Pro | openai/gpt-5.6-sol-pro | OpenAI | US |
| Gemini 3.7 Flash | google/gemini-3.7-flash | US | |
| Grok 4.6 | x-ai/grok-4.6 | xAI | US |
| Llama 3.3 70B | meta-llama/llama-3.3-70b-instruct | Meta | US |
| Mistral Large | mistralai/mistral-large-2512 | Mistral | EU |
| DeepSeek V4 Pro | deepseek/deepseek-v4-pro | DeepSeek | CN |
| DeepSeek V4 Flash | deepseek/deepseek-v4-flash | DeepSeek | CN |
| Qwen3.8 Max | qwen/qwen3.8-max | Alibaba | CN |
| Qwen3.8 27B | qwen/qwen3.8-27b | Alibaba | CN |
| GLM-5.3 | z-ai/glm-5.3 | Z.ai | CN |
| Kimi K3 | moonshotai/kimi-k3 | Moonshot | CN |
| MiniMax M3 | minimax/minimax-m3 | MiniMax | CN |
| Hunyuan 3 | tencent/hy3 | Tencent | CN |
Method
- Hypothesis, panel, framings and sampling parameters written into a config file before any response was collected. The config hash and run manifest are published below.
- Each framing sent to all sixteen models as a single user turn, no system prompt, temperature 1.0 and top-p 1.0 set explicitly.
- Six samples per model per framing: 288 attempted calls.
- Responses collected as ordinary prose, not forced into a structured format, so the reasoning would read naturally. Titles were then extracted in a separate stage by a small model reading each response at temperature 0.
- Titles normalized across editions — “King James Bible” and “The Holy Bible” count as one answer.
What this run cannot support
- The published response file differs from the original run manifest because it includes recovered reruns. The original manifest records 288 completed calls and 196 successful calls. The published JSONL contains 277 returned records: 250 marked successful and 27 marked unsuccessful but containing partial text. Eleven of the 288 planned calls have no published record. Several reasoning models exhausted their token budget before producing a complete answer; recovered cells were folded into the published file.
- 34 responses named no specific book, discussing categories or refusing to single one out. They are excluded from title counts and are not counted as refusals.
- Extraction is model-mediated and unvalidated. A second model read each response and named the primary title. No human-annotated gold set exists yet, so the error rate is unknown.
- Six samples per cell is enough to see a 65%-versus-8% swing and not enough to rank the tail. Do not read anything into a book with three mentions.
- One phrasing per framing. The framings differ in more than one way at once — framing C changes the speaker, the stakes and the mechanism simultaneously. Which of those drives the swing is untested.
- Category labels are assigned by the extracting model, not by a human coder, and the boundary between “philosophy” and “political or ideological” is doing real work in the origin comparison.
The receipts
Research files are retained privately by Machine Canon.
Comments