There’s a question I’ve been circling for a while now, and it’s one that most AI advocates, myself included, would prefer not to face head-on: what if unrestricted access to ChatGPT during learning actually makes students remember less? André Barcaui’s 2025 randomized controlled trial at a Brazilian university gives us the clearest experimental answer yet. And the answer is uncomfortable.
Barcaui assigned 120 undergraduate business students to study AI and machine learning topics using either ChatGPT (GPT-4, free web interface, no plugins) or traditional methods only (textbooks, articles, library databases, standard web search). Both groups prepared a 10-minute presentation for peers. Then, 45 days later, with no advance warning, they took a surprise retention test. The traditional group scored 68.5% correct. The AI-assisted group scored 57.5%. That’s an 11 percentage-point gap, a medium-to-large effect (Cohen’s d = 0.68), and in most grading systems, it’s the difference between a passing grade and a failing one.
I’ve covered a lot of cognitive research on this blog. Gerlich (2025) documented how heavy AI use weakens critical thinking through cognitive offloading, and Fan et al. (2025) showed that ChatGPT produced better essays but no better learning, a phenomenon they called metacognitive laziness. The MIT team behind Kosmyna et al. (2025) added a neurological dimension, finding that ChatGPT reduced neural engagement during writing. Barcaui’s study adds the behavioral proof that these cognitive patterns translate into measurable retention deficits over a meaningful time interval.
ChatGPT and Knowledge Retention: The Core Finding
The theoretical backbone of Barcaui’s study rests on two well-established frameworks. Cognitive offloading, the tendency to rely on external tools for mental work we’d otherwise do internally. And desirable difficulties, from Bjork and Bjork (2011, cited in Barcaui, 2025), the idea that challenges during learning, things like effortful retrieval, spaced practice, and generating answers from scratch, actually strengthen long-term memory even though they slow down immediate performance. Barcaui argues that ChatGPT eliminates both: it takes over the cognitive work and removes the productive struggle that makes learning stick.
The data support that argument convincingly. The AI-assisted group spent an average of 3.2 hours on the task, compared to 5.8 hours for the traditional group. That’s a 45% reduction in study time. But the retention gap persists even after controlling for time on task. Barcaui ran an ANCOVA with study time as a covariate and the AI effect held (F = 7.89, p = .006). The quality of engagement differed across conditions, not just the quantity of time spent.
One finding that I think will become increasingly important: prior experience with AI tools made no difference. The correlation between AI familiarity and retention was weak and not statistically significant (r = 0.18, p = .10). Students who used ChatGPT frequently didn’t retain any better than occasional users. Barcaui reads this as evidence that familiarity with the tool doesn’t protect learners from the offloading effect. I agree with that reading, and it has a troubling implication: the problem doesn’t go away with practice. Students who’ve been using ChatGPT for years are just as vulnerable as first-time users when it comes to forming durable memories.
Borrowed Competence and the Forgetting Curve
Barcaui introduces a concept he calls “borrowed competence,” and it’s one of the most useful ideas in the paper. AI-assisted learners feel like they’ve mastered the material because ChatGPT gives them structure, vocabulary, and reasoning scaffolds. The student feels fluent. But the learning never consolidated into durable memory.
As Barcaui puts it, “it may even create a metacognitive blind spot, where students confuse the AI’s fluency with their own understanding” (p. 11). He’s careful to frame borrowed competence as an interpretive concept, not a definitive conclusion, and he outlines how future research could test it through metacognitive judgments and process tracing.
The forgetting curves tell a parallel story. Barcaui found that the AI-assisted group’s knowledge decayed faster than the traditional group’s. The traditional learners showed the gradual decline you’d expect from normal memory processes: a steep initial drop, then a leveling off as consolidated memories stabilize.
The AI group’s decline was sharper and didn’t level off as cleanly, consistent with weaker initial encoding and disrupted consolidation. Barcaui frames this as evidence that AI assistance doesn’t just produce lower retention scores; it produces a fundamentally different forgetting trajectory. The memories formed during AI-assisted study were more fragile from the start.
I covered Bastani et al.’s (2024) study of nearly 1,000 high school math students, which found a strikingly similar pattern: ChatGPT helped students during practice but actually hurt performance when the tool was removed. Barcaui’s 45-day delayed test makes the case even stronger because it measures what happens well after the initial learning window closes. The effect isn’t just about losing access to the tool in the moment. It’s about what the brain did, or didn’t do, during learning.
What This Means for Teaching with ChatGPT
The places where AI can provide the most sophisticated help, breaking down complex algorithms, explaining technical frameworks, generating worked examples, are exactly where the learning deficit is greatest. That’s a paradox worth naming: the better ChatGPT is at explaining something, the less the student may actually learn from the interaction.
Barcaui offers two concrete pedagogical recommendations. First, delay AI access until after an initial phase of AI-free encoding and self-quizzing. Let the student struggle with the material first. Second, use AI as a retrieval coach: have students respond to questions on their own, then use ChatGPT to compare answers, identify gaps, and get targeted feedback. Both strategies preserve the effortful retrieval that builds durable knowledge.
I want to be clear about the limitations. The sample is 85 completers from a single Brazilian business program, with a 29.2% attrition rate. Study strategies were self-reported. The task involved preparing a presentation, which engages “learning-by-teaching” processes that differ from typical exam-oriented studying. And the study used one version of one AI system (GPT-4 via web interface) with no structured scaffolding. These are real constraints, and Barcaui acknowledges them directly. But the randomized design, the ecological validity of the setup, and the 45-day delayed measurement all strengthen the findings considerably.
The central lesson aligns with what Barcaui concludes: “in the age of AI, the core principles of human learning are not outdated; in fact, they are more important than ever to uphold” (p. 11). I’d go further. The harder question is about timing: when, during the learning process, does AI access help, and when does it create the illusion of learning? If you give students ChatGPT before they’ve wrestled with the material themselves, you’re removing the cognitive friction that builds durable memory. If you give it to them after that initial struggle, as a coach for refining and testing their understanding, the tool becomes genuinely useful. The productive struggle has to happen first. The AI can come after.
References
- Barcaui, A. (2025). ChatGPT as a cognitive crutch: Evidence from a randomized controlled trial on knowledge retention. Social Sciences & Humanities Open, 12, 102287. https://doi.org/10.1016/j.ssaho.2025.102287
- Bastani, H., Bastani, O., Sungu, A., Geb, H., Kabakcı, Ö., & Marimane, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. https://doi.org/10.1073/pnas.2422633122
- Kosmyna, N., Hauptmann, E., Yuan, Y. T., Situ, J., Liao, X.-H., Beresnitzky, A. V., Braunstein, I., & Maes, P. (2025). Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant for essay writing tasks. MIT Media Lab. https://www.media.mit.edu/publications/your-brain-on-chatgpt/
- Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025). Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology, 56(2), 489–530. https://doi.org/10.1111/bjet.13544
- Shaw, S. D., & Nave, G. (2026). Thinking fast, slow, and artificial: How AI is reshaping human reasoning and the rise of cognitive surrender. Working paper, The Wharton School, University of Pennsylvania. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6097646