Prompt Engineering Challenge
Demonstrate the difficulty of achieving reproducible AI outputs, highlighting the inherent variability in AI systems.
Recommended for: Introduction to prompt engineering principles
Procedure
- Provide students with a single article or text passage
- Challenge students to craft prompts that will produce identical summaries across multiple attempts with the same AI system
- Allow students to iterate on their prompts, testing various formulations
- Discuss the results and whether achieving true reproducibility is possible
Discussion questions
- Why is reproducibility important in academic and scientific work?
- What does the observed variability in AI outputs suggest about their reliability for scholarly purposes?
- How does AI output reproducibility compare to that of traditional academic tools and methods?
Assessment opportunity
Students submit their most successful prompt along with a brief analysis of what made reproducibility challenging.