Exercise: AI scavenger hunt

Recommended for: Early and upper undergraduate; adaptable to any discipline

Background for the instructor

A persistent finding in the educational literature on AI is that students who have not been explicitly trained tend to over-trust AI-generated content, particularly when it is formatted to resemble scholarly writing (Sundar & Liao, 2023).

The scavenger hunt format addresses over-trust through discovery learning: students who successfully identify an AI-generated text by finding its flaws are likely to internalise the lesson more durably than students who are simply told that AI makes errors. The group format is important because different students notice different markers, and the discussion itself builds shared criteria for source evaluation.

Preparation

1. Before the session, the instructor prepares a set of three to five texts on a topic relevant to the course. At least one text should be generated by AI (without post-editing). The remaining texts should be drawn from a range of real sources: a peer-reviewed abstract, a news article, a textbook excerpt, and optionally a student-written text. The AI-generated text should not be identified to students in advance. Remove author names and publication details from all texts so that source recognition cannot substitute for close reading.

Procedure

1. Distribute the text set to small groups of three to four students. Groups do not need to receive identical sets, though overlapping sets make the plenary discussion richer.
2. Groups read all texts and discuss: which text or texts are likely to have been generated by AI, and which markers led them to that conclusion. Ask groups to document their reasoning explicitly, not just their verdict.
3. Groups also assess the reliability of each source on a three-point scale (reliable, uncertain, unreliable) and note the criteria they applied.
4. In the plenary, groups share their conclusions. Reveal which text was AI-generated. Discuss cases where groups disagreed.
5. The instructor shares a short list of tools and habits for source evaluation relevant to the discipline (for example, checking journal databases, verifying author credentials, cross-checking specific claims).
6. Close with the discussion questions below.

Discussion questions

• What markers, if any, distinguished the AI-generated text from the others? Were these markers about content, style, or both?
• Were there real texts that were harder to assess than the AI-generated one? What does that suggest about the reliability of stylistic markers alone?
• If you had not been told that one text was AI-generated, would you have spotted it in your regular coursework reading? What would need to change in your reading habits?
• What is the difference between a text that reads reliably and a text that is reliable? How do you bridge that gap in practice?