There’s a question I’ve been circling for a while now, and it’s one that most AI advocates, myself included, would prefer not to face head-on: what if unrestricted access to ChatGPT during learning actually makes students remember less? André Barcaui’s 2025 randomized controlled trial at a Brazilian university gives us the clearest experimental answer yet. And the answer is uncomfortable.
Barcaui assigned 120 undergraduate business students to study AI and machine learning topics using either ChatGPT (GPT-4, free web interface, no plugins) or traditional methods only (textbooks, articles, library databases, standard web search). Both groups prepared a 10-minute presentation for peers. Then, 45 days later, with no advance warning, they took a surprise retention test. The traditional group scored 68.5% correct. The AI-assisted group scored 57.5%. That’s an 11 percentage-point gap, a medium-to-large effect (Cohen’s d = 0.68), and in most grading systems, it’s the difference between a passing grade and a failing one.
I’ve covered a lot of cognitive research on this blog. Gerlich (2025) documented how heavy AI use weakens critical thinking through cognitive offloading, and Fan et al. (2025) showed that ChatGPT produced better essays but no better learning, a phenomenon they called metacognitive laziness. The MIT team behind Kosmyna et al. (2025) added a neurological dimension, finding that ChatGPT reduced neural engagement during writing. Barcaui’s study adds the behavioral proof that these cognitive patterns translate into measurable retention deficits over a meaningful time interval.
ChatGPT and Knowledge Retention: The Core Finding
The theoretical backbone of Barcaui’s study rests on two well-established frameworks. Cognitive offloading, the tendency to rely on external tools for mental work we’d otherwise do internally. And desirable difficulties, from Bjork and Bjork (2011, cited in Barcaui, 2025), the idea that challenges during learning, things like effortful retrieval, spaced practice, and generating answers from scratch, actually strengthen long-term memory even though they slow down immediate performance. Barcaui argues that ChatGPT eliminates both: it takes over the cognitive work and removes the productive struggle that makes learning stick.
The data support that argument convincingly. The AI-assisted group spent an average of 3.2 hours on the task, compared to 5.8 hours for the traditional group. That’s a 45% reduction in study time. But the retention gap persists even after controlling for time on task. Barcaui ran an ANCOVA with study time as a covariate and the AI effect held (F = 7.89, p = .006). The quality of engagement differed across conditions, not just the quantity of time spent.
One finding that I think will become increasingly important: prior experience with AI tools made no difference. The correlation between AI familiarity and retention was weak and not statistically significant (r = 0.18, p = .10). Students who used ChatGPT frequently didn’t retain any better than occasional users. Barcaui reads this as evidence that familiarity with the tool doesn’t protect learners from the offloading effect. I agree with that reading, and it has a troubling implication: the problem doesn’t go away with practice. Students who’ve been using ChatGPT for years are just as vulnerable as first-time users when it comes to forming durable memories.
Borrowed Competence and the Forgetting Curve
Barcaui introduces a concept he calls “borrowed competence,” and it’s one of the most useful ideas in the paper. AI-assisted learners feel like they’ve mastered the material because ChatGPT gives them structure, vocabulary, and reasoning scaffolds. The student feels fluent. But the learning never consolidated into durable memory.
As Barcaui puts it, “it may even create a metacognitive blind spot, where students confuse the AI’s fluency with their own understanding” (p. 11). He’s careful to frame borrowed competence as an interpretive concept, not a definitive conclusion, and he outlines how future research could test it through metacognitive judgments and process tracing.
The forgetting curves tell a parallel story. Barcaui found that the AI-assisted group’s knowledge decayed faster than the traditional group’s. The traditional learners showed the gradual decline you’d expect from normal memory processes: a steep initial drop, then a leveling off as consolidated memories stabilize.
The AI group’s decline was sharper and didn’t level off as cleanly, consistent with weaker initial encoding and disrupted consolidation. Barcaui frames this as evidence that AI assistance doesn’t just produce lower retention scores; it produces a fundamentally different forgetting trajectory. The memories formed during AI-assisted study were more fragile from the start.
I covered Bastani et al.’s (2024) study of nearly 1,000 high school math students, which found a strikingly similar pattern: ChatGPT helped students during practice but actually hurt performance when the tool was removed. Barcaui’s 45-day delayed test makes the case even stronger because it measures what happens well after the initial learning window closes. The effect isn’t just about losing access to the tool in the moment. It’s about what the brain did, or didn’t do, during learning.
How to Use ChatGPT Without Removing the Learning
The study does not tell educators to remove ChatGPT from learning. It points to the importance of timing and instructional design. Students may need an initial period in which they read, retrieve information, attempt an explanation, and identify what they do not understand before receiving extensive help from AI.
One practical approach is to delay access. Ask students to examine the material and produce an initial response first. ChatGPT can then help them compare explanations, identify gaps, generate additional questions, or receive feedback on what they have already attempted.
Teachers can preserve the cognitive work of learning by asking students to:
- answer a question before consulting ChatGPT;
- explain a concept from memory and then check the explanation;
- predict an answer before asking AI for help;
- verify the AI response against course readings;
- identify errors or omissions in an AI-generated explanation;
- close the chatbot and retrieve the information again later;
- apply the concept to a new example without AI assistance.
ChatGPT may also be useful as a retrieval coach. It can generate practice questions, withhold answers until the student responds, and provide feedback based on the student’s attempt. The student continues to retrieve and explain information while the AI supports the practice.
The limitations of the trial matter. Only 85 students completed the study, all from a single Brazilian business program. Study practices were self-reported, and the task involved preparing a presentation. The ChatGPT group also used one version of GPT-4 without structured instructional guardrails. We cannot assume that every form of guided AI use would produce the same result.
Even with those limitations, the 45-day delayed test makes this study difficult to dismiss. Students using ChatGPT completed the work in considerably less time, but their later retention was lower. Efficiency during a task and learning from that task are separate outcomes.
For me, the central question is therefore when students encounter AI during the learning process. If ChatGPT provides the explanation before students have tried to understand the material, it can remove some of the work that helps knowledge stick. When students attempt the task first, AI can help them test, refine, and extend what they have begun to understand.
References
- Barcaui, A. (2025). ChatGPT as a cognitive crutch: Evidence from a randomized controlled trial on knowledge retention. Social Sciences & Humanities Open, 12, 102287. https://doi.org/10.1016/j.ssaho.2025.102287
- Bastani, H., Bastani, O., Sungu, A., Geb, H., Kabakcı, Ö., & Marimane, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. https://doi.org/10.1073/pnas.2422633122
- Kosmyna, N., Hauptmann, E., Yuan, Y. T., Situ, J., Liao, X.-H., Beresnitzky, A. V., Braunstein, I., & Maes, P. (2025). Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant for essay writing tasks. MIT Media Lab. https://www.media.mit.edu/publications/your-brain-on-chatgpt/
- Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025). Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology, 56(2), 489–530. https://doi.org/10.1111/bjet.13544
- Shaw, S. D., & Nave, G. (2026). Thinking fast, slow, and artificial: How AI is reshaping human reasoning and the rise of cognitive surrender. Working paper, The Wharton School, University of Pennsylvania. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6097646
