I’ve been talking with teachers about AI for two years now, and one pattern shows up in almost every conversation. We slip into anthropomorphic language without noticing. The AI “thinks,” the AI “knows,” the AI “decides,” the AI “understands.” It feels natural.
ChatGPT writes like a person, so we talk about it like a person. Murray Shanahan’s 2024 paper in Communications of the ACM, “Talking About Large Language Models,” is a careful philosophical argument against this habit. It’s an older paper now in AI time, but the argument matters more as systems get more convincing.
What Shanahan Actually Argues
Shanahan’s central claim is straightforward. An LLM is a generative model of the statistical distribution of tokens in human text. When you prompt it, you’re not asking it a question in the way you’d ask a friend. You’re asking the model what words are most likely to follow the sequence you gave it.
When ChatGPT answers “Who walked on the moon first?” with “Neil Armstrong,” it’s not because the model knows about Apollo 11. It’s because that sequence is highly likely in its training corpus. The output looks like knowledge. The mechanism is not.
Shanahan is careful to separate the bare-bones LLM (the trained model itself) from the larger system it’s typically embedded in (dialog manager, tools, retrieval, fine-tuning). A lot of confusion in popular AI discourse comes from collapsing these two together. The model alone is one thing. The product you interact with is something more.
The Category Mistake
The sharpest move in the paper is the category mistake argument. Shanahan asks: does an LLM that produces “Burundi is south of Rwanda” know that fact?
His answer is no, at least for the bare-bones model. The model knows that the word “Burundi” is statistically likely to follow the sequence “the country south of Rwanda is.” That’s a fact about the distribution of tokens in English text. It’s not the same as a fact about geography.
Shanahan argues that this confusion is “a profound category mistake.” We treat the model’s output as if it were a claim about the world. The model has no concept of the world. It has a concept of token sequences.
As he puts it, “Interacting with a contemporary LLM-based conversational agent can create a compelling illusion of being in the presence of a thinking creature like us. Yet, in their very nature, such systems are fundamentally not like us” (p. 78).
AI Anthropomorphism in the Classroom
For me as an educator, the paper’s most useful contribution is the diagnostic it gives us for AI literacy. When teachers or students say “the AI thinks X” or “the AI knows Y,” that’s anthropomorphic shorthand. Some uses are harmless. Others are misleading.
The misleading cases matter for the classroom. A student who treats ChatGPT as a knowing entity is likely to accept its output without checking, miss the cases where it confidently invents citations, and overestimate what the system can do. A teacher who frames AI as a “thinking assistant” makes that same mistake easier to fall into.
In my elementary AI Use Agreement, I included a small section called “AI Is Not a Person” specifically to address this. Young children anthropomorphize AI even faster than adults do. The fix isn’t to forbid all human language about AI. It’s to keep the underlying mechanism in mind, so the shorthand doesn’t become a misunderstanding.
This connects to Liu et al.’s (2026) recent finding that how students treat AI shapes the cognitive outcome. Students who used AI for direct answers lost persistence; students who used it for hints didn’t. The framing matters because it shapes behavior.
Shanahan’s framing also connects to Roe, Furze, and Perkins’s (2025) “digital plastic” metaphor work, where they argue that the metaphors we use for AI shape how students engage with it. It also fits Kalantzis and Cope’s (2025) broader argument that literacy in the time of AI requires understanding what AI actually does.
Limitations
Shanahan’s core argument is sound. The next-token-prediction frame is the right one to keep in mind, and the warning about anthropomorphism is correct.
Where I’d push: the paper was written in late 2023, when GPT-4 was the frontier. Two years on, the picture has moved. Agentic systems with tools, retrieval, and multimodal grounding are standard. Reasoning-trained models like o1 and o3 are explicitly designed to do something closer to faithful inference than pure pattern completion. The bare-bones LLM still does what Shanahan describes, but the products students actually interact with are more complex systems. Some of his “what if we embed the LLM in…?” hypotheticals are now mainstream practice.
This doesn’t undermine the paper. It actually strengthens it. As systems get more capable, the anthropomorphism trap gets stronger. Shanahan warns that “The careless use of philosophically loaded words such as ‘believes’ and ‘thinks’ is especially problematic, because such terms obfuscate mechanism and actively encourage anthropomorphism” (p. 78). That’s truer in 2026 than it was in 2024.
The practical question for teachers is what to do about it. My answer is the one I keep landing on: AI literacy starts with knowing what’s happening under the hood, even if we don’t use technical language with younger students. Talking about AI accurately is itself a form of pedagogy.
References
- Kalantzis, M., & Cope, B. (2025). Literacy in the time of artificial intelligence. Reading Research Quarterly, 60, e591. https://doi.org/10.1002/rrq.591
- Liu, G., Christian, B., Dumbalska, T., Bakker, M. A., & Dubey, R. (2026). AI assistance reduces persistence and hurts independent performance [Preprint]. arXiv. https://arxiv.org/abs/2604.04721. https://medkharbach.com/ai-and-persistence-the-cognitive-cost-of-quick-answers/
- Roe, J., Furze, L., & Perkins, M. (2025). Digital plastic: A metaphorical framework for Critical AI Literacy in the multiliteracies era. Pedagogies: An International Journal. Advance online publication. https://doi.org/10.1080/1554480X.2025.2557491
- Shanahan, M. (2024). Talking about large language models. Communications of the ACM, 67(2), 68-79. https://doi.org/10.1145/3624724