Summary Variation Analysis
Reveal the variability and inconsistency in AI-generated outputs, helping students understand that AI systems do not produce uniform or necessarily reliable results.
Recommended for: First exposure to AI tools
Procedure
- Assign all students the same scholarly article appropriate to course content
- Instruct students to generate summaries using different generative AI systems (e.g., ChatGPT, Claude, Gemini, Copilot)
- Have students compare their AI-generated summaries with those of neighboring classmates
- Facilitate a structured discussion on variations in content, emphasis, accuracy, and structure across different AI platforms
Discussion questions
- What factors might account for the differences between AI-generated summaries?
- What are the implications of this variability for academic reliability and consistency?
- Under what circumstances, if any, can AI-generated content be considered trustworthy?
Assessment opportunity
Students document three substantive differences between summaries and hypothesize reasons for these variations.