Critical AI interaction in writing involves more than asking ChatGPT for information or cleaner sentences. Students need to judge what the system gives them, follow up when an answer is weak, verify claims, and remain responsible for the direction of the work. Those actions are easy to recommend and much harder to see in practice.
A study by Antonette Shibani and colleagues gives us a closer look at that practice through student prompts and reflections. It belongs within the wider conversation on AI and assessment because it shifts attention from whether students used AI to how they engaged with it while completing assessed writing.
Summary of the Study
The researchers developed the Critical Interaction with AI for Writing (CIAW) framework by combining models of writing, online information problem-solving, and deep and surface approaches to learning. The framework examines five parts of AI-supported writing: planning and ideation, information seeking and evaluation, writing and presentation, personal reflection, and conversational engagement.
Each part is classified as deep, shallow, or absent. The aim is to describe whether students actively questioned and developed AI output or used the tool in a largely transactional way.
The team applied the framework to assignments from 49 graduate data science students at an Australian university. Students wrote a critical summary about an ethical issue in natural language processing, reflected on their use of AI, and submitted their ChatGPT exchanges when they had used the tool. Most of the visible interactions were shallow.
Students commonly asked for starting ideas, explanations, sources, rephrasing, formatting help, or direct information. Few prompts showed sustained dialogue, careful evaluation, or substantial revision of the system’s response. No deep interaction was identified in the writing and presentation dimension.
The reflections told a somewhat different story. Some students recognized fabricated references, factual errors, overreliance, and the possible loss of original thought. This suggests that learners can notice AI’s weaknesses without consistently turning that awareness into stronger prompting or writing decisions.
The gap matters. Knowing that an output should be checked is not the same as checking it, just as describing critical thinking is not evidence that it occurred. This is why evaluative judgement in the AI age needs to be made visible through decisions such as verifying a source, rejecting a claim, or revising an answer for a disciplinary purpose.
The study is exploratory, and the framework should be treated accordingly. It examines one assignment in one graduate course, using a subset of submissions from students who supplied prompts and reflections. Those records may be incomplete, and they do not capture students’ private reasoning, searches conducted elsewhere, or conversations they chose not to submit.
The coding also gives precedence to evidence of deep interaction when prompts and reflections conflict, which can make a student’s overall engagement appear stronger than the prompt trail alone suggests. CIAW offers a promising language for examining AI-supported writing, but it still needs testing across subjects, tasks, student groups, and newer AI systems.
Practical Insights
The most useful lesson is that permission to use AI does not automatically produce critical engagement. Students need an assessment design that asks them to show what they did with the output.
- Require decision traces. Ask students to submit a small number of consequential exchanges and explain what they accepted, changed, rejected, and verified. A complete transcript can create unnecessary volume; selected moments with commentary reveal more about judgment.
- Teach follow-up moves. Give students an initial AI response and ask them to challenge an assumption, request missing evidence, test a counterargument, or adapt the answer for a specific audience. Research on asking AI better questions shows why prompt quality deserves direct instruction, although stronger prompts must still be paired with evaluation.
- Connect reflection to evidence. Replace general statements such as “I checked the answer” with prompts that ask what was checked, where, what was found, and how the draft changed. This makes reflection harder to perform as a ritual at the end of the assignment.
- Separate stages of the writing process. Students can record how AI contributed to planning, information seeking, drafting, and revision. Teachers can then see whether the tool supported a stage or quietly performed it for the student.
- Use the framework diagnostically. CIAW can help teachers review an activity and identify where it invites only one-shot commands. It should not yet be treated as a settled scoring instrument or a shortcut for judging an individual student’s critical-thinking ability.
Future research could combine prompt records with screen capture, version histories, interviews, and analysis of the final text. It should also examine whether deeper interaction leads to better reasoning, stronger writing, or more independent performance after AI support is removed. Without those outcomes, an elaborate conversation with a chatbot may look intellectually active while adding little to learning.
The study leaves educators with a productive shift in focus. An AI-use declaration tells us that a tool entered the process. The more important evidence lies in the student’s decisions: where the system was questioned, how its output was tested, and what intellectual responsibility remained with the writer.
Reference
Shibani, A., Knight, S., Kitto, K., Karunanayake, A., & Buckingham Shum, S. (2024). Untangling critical interaction with AI in students’ written assessment. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’24) (Article 357, pp. 1–6). Association for Computing Machinery. https://doi.org/10.1145/3613905.3651083
