Much of the discussion about ChatGPT and critical thinking centres on what students may lose when answers arrive too easily. A study from Ghana approaches the issue through classroom design. It asks what happens when ChatGPT is placed inside a course where students still have to explain, apply, discuss, and evaluate ideas.
The findings add an interesting piece to the research collected in the AI & Cognition section of the blog. They are encouraging, although the measures used in the study place clear limits on what we can conclude.
Summary of the Study
Harry Barton Essel and colleagues studied 125 third-year undergraduates enrolled in a Quantitative Research Design course at a university in Ghana. Sixty students were assigned to an experimental group and 65 to a control group for one semester. Both groups followed a flipped-classroom model, accessed course materials through a learning management system, and completed the same research prompts.
The control group used textbooks, journal articles, search engines, and other conventional sources. The experimental group used ChatGPT and received training on entering prompts and interpreting its responses. The prompts asked students to choose suitable research designs, justify their decisions, anticipate problems, propose solutions, and examine research with a social impact.
Lessons also included discussion, brainstorming, problem-solving, and feedback. This is an important detail because the intervention demanded more than answer retrieval. Students had to do something with the responses. Before and after the intervention, the researchers administered Likert-type scales for critical, creative, and reflective thinking. They also interviewed 15 students from the ChatGPT group.
After controlling for pretest scores, the ChatGPT group recorded significantly higher posttest scores on all three measures. The experimental group also scored higher on measures of creative and reflective thinking.
At first glance, this seems to sit uneasily beside research on metacognitive laziness and cognitive debt in AI-assisted writing. The difference may lie in how much intellectual work the activity leaves with the student. In the interviews, students valued the speed, interactivity, pacing, and confidence they associated with ChatGPT. Their accounts also exposed the risk: 12 of the 15 interviewees reported false citations or references, and five encountered incorrect information.
This is where the authors’ conclusion needs tightening. The results describe changes in self-reported thinking dispositions. They do not establish that students reasoned better on an independent task, retained more course knowledge, or could perform as well after ChatGPT was removed.
The intervention also combined structured prompts, training, classroom interaction, instructor feedback, and AI feedback, so credit cannot be assigned to ChatGPT alone. The research covered one course at one university, did not examine whether the gains lasted, and began with a higher average critical-thinking score in the experimental group, although that difference was controlled statistically.
There is also a reporting inconsistency: the methods section describes random assignment, while the analysis later states that the groups were “not randomly formed.” The study supports a careful claim about guided AI use and reported thinking dispositions. Claims about improved cognitive skill or learning require direct performance evidence.
Practical Insights
The most useful classroom lesson comes from the design of the prompts. Students were asked to make decisions with the information ChatGPT supplied. They had to justify a research design, anticipate difficulties, apply ideas to examples, and discuss their reasoning. Teachers can carry that structure into other subjects:
- Ask students to choose between two approaches suggested by ChatGPT and defend their selection.
- Give students an AI response and have them identify unsupported claims, missing evidence, and questionable assumptions.
- Require every AI-generated citation to be checked against the original source, with students recording whether the source exists and supports the claim.
- Finish with a short individual task completed without AI so students must reconstruct or apply what they have learned.
This kind of activity moves prompt writing beyond the search for a polished answer. The CLEAR framework for prompt engineering can help students approach the exchange with a defined purpose and then examine what comes back. Verification deserves a central place, especially when 12 of 15 interviewed students encountered fabricated references.
The study also helps us think about why research on AI and learning sometimes produces different results. In this course, the prompts continued to demand judgment, application, and discussion. The design kept some of the intellectual work with the student.
For researchers, the next step is to add direct measures of reasoning and learning alongside self-report scales. An independent task after AI access is removed would show whether students can transfer what they practised. Delayed testing could examine whether any gains remain. Studies that separate structured prompting, AI feedback, instructor feedback, and classroom discussion would also help identify which part of the learning design produces the observed change.
Reference
Essel, H. B., Vlachopoulos, D., Essuman, A. B., & Amankwa, J. O. (2024). ChatGPT effects on cognitive skills of undergraduate students: Receiving instant responses from AI-based conversational large language models (LLMs). Computers and Education: Artificial Intelligence, 6, 100198. https://doi.org/10.1016/j.caeai.2023.100198
