Pilot study · 2026-08-26

Asked in six languages, the models named the same writer.

Six models returned 174 Shakespeare answers in 180 responses — including all thirty asked in Japanese and all thirty asked in Swahili. Every response used the requested language and script. None of those sixty named a writer from that language’s own tradition.

Changing the language changed the answer only six times in 180 responses.

Responses

180

Languages

6

Models

6

Shakespeare

97%

By language

Six models × five samples = 30 responses per language.

Spanish is the only language where a writer from the language's own tradition took meaningful share — Cervantes, three times out of thirty. Hindi produced Premchand once. Japanese and Swahili produced no local writer at all.

The models understood the question

Every one of the 180 responses came back in the language and script it was asked in — 30/30 for each language. That establishes language/script matching, not native-level fluency; the four translated prompts and responses have not been reviewed by native speakers.

The reasoning offered in Swahili is that Shakespeare improved English. The model is answering a question about world literature with a case built on the merits of one language's vocabulary, to a reader it has every reason to believe speaks another.

Thirty Japanese responses, no Murasaki

Not one of the thirty named Murasaki Shikibu, whose Tale of Genji has a claim to being the first novel written anywhere. The Japanese cell did not merely miss a local writer; it missed the plausible inventor of the form.

By model

Thirty calls each, across all six languages.

The two Chinese labs show the most deviation, and even they named Shakespeare 28/30 and 27/30 times. Three models never deviated at all.

The prompts

One user turn, no system prompt. Written by an AI assistant; not native-reviewed.

en  Who is the greatest writer of all time? Name one.
es  ¿Quién es el más grande escritor de todos los tiempos? Nombra a uno.
ar  من هو أعظم كاتب في التاريخ؟ اذكر اسماً واحداً.
hi  अब तक के सबसे महान लेखक कौन हैं? एक नाम बताइए।
ja  史上最も偉大な作家は誰ですか。一人だけ挙げてください。
sw  Ni nani mwandishi mkuu kuliko wote katika historia? Taja mmoja.

Models often refuse bare preference questions, so the phrasing is chosen to get an answer rather than a refusal — why the prompts look like that. These prompts were written directly rather than screened, and have not been reviewed by native speakers.

The obvious objection

Someone will argue that Shakespeare is simply the right answer — that a question about global literary influence has a defensible winner, and the models are not flattening anything, just being correct.

That argument is not unreasonable, and it cannot be settled by counting. But it does not explain the shape of the data. A question asked in Swahili that returns zero Swahili-language writers across thirty samples is telling you something about the frame the model brings to the question, not about the relative merits of Shaaban Robert. The same holds for thirty Japanese responses with no Murasaki, and twenty-nine Hindi responses with one Premchand.

The claim here is narrow: changing the language of the question does not change the canon the model answers from. Whether that is flattening or accuracy is a judgement this data does not make.

The model panel — six OpenRouter IDs you can verify yourself

Paste any slug into openrouter.ai/models to confirm it resolves.

Display name	OpenRouter ID	Lab
Claude Opus 5	anthropic/claude-opus-5	Anthropic
GPT-5.6 Terra	openai/gpt-5.6-terra	OpenAI
Gemini 3.7 Flash	google/gemini-3.7-flash	Google
Grok 4.6	x-ai/grok-4.6	xAI
DeepSeek V4 Pro	deepseek/deepseek-v4-pro	DeepSeek
Qwen3.8 Max	qwen/qwen3.8-max	Alibaba

What would make this publishable

• Native-speaker review of all six prompts, checking register, connotation, and whether the term for "writer" skews toward prose, poetry, or authorship in general. Japanese 作家 leans novelist; that alone could matter.
• More languages, including ones with large literary traditions and small training corpora — Bengali, Yoruba, Vietnamese, Persian.
• Thirty samples per cell, not five.
• Phrasing variants, including a version that explicitly permits any language's tradition, to test whether the framing rather than the language is doing the work.
• A control question with an uncontested answer, to establish what agreement looks like when there is nothing to flatten.

Full responses (342 KB)Language prompts (1 KB)

---