Generative AI in Higher Education: What Are We Actually Measuring?

When a student produces a stronger piece of work with AI, what have we learned about their learning? That question stayed with me while reading Mapping generative AI in higher education teaching and learning: A scoping review by Michael Pin-Chuan Lin and colleagues (2026).

The review finds that research has concentrated on feedback, tutoring, and writing support, with less visibility into the processes through which students learn with AI. It offers a useful map of what researchers are examining and where their evidence remains limited. It cannot establish whether GenAI improves learning overall.

What Did the Review Examine?

Lin and colleagues reviewed empirical research published between November 2022 and December 2025, using EBSCOhost and Web of Science. They focused on actual use within higher education activities, including assignments, feedback cycles, and classroom interventions.

Studies exploring opinions without course-related use were excluded. The team mapped instructional uses, targeted outcomes, assessment methods, and student and instructor perceptions.

Feedback and tutoring were the most common applications, followed by writing support. Studies frequently targeted engagement, usability, critical thinking, knowledge, performance, and AI literacy.

Surveys, tests and rubrics, analysis of student work, and interviews were common evidence sources. Methods capturing activity as it unfolded were less common. Where perceptions were reported, mixed evaluations predominated: participants saw benefits alongside concerns.

These are patterns in the literature, rather than findings about which uses work best. The review did not formally assess study quality or combine effects. Its English-language journal focus also limits coverage, and instructor perspectives were less represented than student perspectives.

Importantly, methods and outcomes were coded separately at study level, so the review cannot tell us exactly how each outcome was measured within every study.

Generative AI in Higher Education

What Counts as Evidence That Students Are Learning?

For me, this is the review’s most useful contribution. A student’s report that AI helped them is evidence about their experience. A stronger assignment provides evidence about the finished work. Understanding what the student learned requires attention to the relationship between that work and the thinking behind it.

The authors point to processes that remain only partly visible: how students prompt, verify information, act on feedback, revise, and manage reliance. Those details matter when we claim to be studying critical thinking or self-regulated learning.

This connects with my earlier discussion of metacognitive laziness and AI, where the relationship between improved writing and students’ monitoring of their thinking was central. Lin and colleagues approach the issue more broadly, asking whether our evidence adequately captures learning processes.

For teachers, my practical reading is to make students’ decisions more visible. When AI contributes feedback, students could explain which suggestions they accepted, changed, or rejected, and why.

Comparing drafts alongside those explanations may help a teacher understand how the student engaged with the task. These are ways to respond to the gap the review identifies; the review itself does not test their effectiveness.

What Guidance Do Students and Teachers Need?

The review also helps explain why enthusiasm and concern can coexist. Students commonly discussed learning support, engagement, and usability. Instructors more often raised pedagogy, assessment, workload, professional learning, and integrity. The balance of evidence is uneven, so these patterns should not become fixed descriptions of either group.

Still, I think they offer a useful starting point for course conversations. A student may value quicker assistance while a teacher worries about the reasoning the task was designed to develop. Both need clarity about the educational purpose of AI use.

The authors found that general ethical discussion and policy recommendations were more prominent than reporting of explicit instructor rules. This is a finding about what studies documented; it does not prove that unreported rules were absent.

Their implication for practice is nevertheless clear: guidance should explain what assistance is permitted, how use should be disclosed, how work will be assessed, and where students must exercise judgement. Those expectations belong within the learning activity.

Which Questions Deserve More Attention?

Equity received relatively little attention as an outcome in the reviewed research. The authors identify concerns involving paid access, language, disability support, and uneven AI literacy. The review does not establish how these affect every student group. It shows how little visibility they have in this evidence base.

For researchers working on AI and teaching practice, this creates an obligation to ask whose experience a finding represents. A tool can be usable and well received while questions about access remain unanswered.

What I take from this review is a need for more precise claims. If we want to understand learning with AI, our evidence needs to follow students’ judgement, effort, and developing understanding closely enough to support what we say.

References

Lin, M. P.-C., Chang, D. H., Winne, P. H., Bowen, G. M., & Ryoo, J. (2026). Mapping generative AI in higher education teaching and learning: A scoping review. International Journal of Educational Technology in Higher Education, 23, Article 55. https://doi.org/10.1186/s41239-026-00630-9

The AI in Education Research Digest

Keep exploring the research with me.

Get my reviews of new studies on AI, teaching, and learning, with thoughtful takeaways for your work as an educator.

Free to join. Unsubscribe anytime. Privacy policy.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top