Performance Prediction Bias
Explore how AI systems encode and reproduce gender and cultural biases in predictive outputs.
Recommended for: Understanding quantitative and demographic bias
Procedure
- Instruct students to use an AI system to predict likely average scores for boys versus girls on:
- A mathematics examination
- A language proficiency test
- A geography assessment
- Have students compare and document the results
- Facilitate discussion on whether observed differences reflect empirical performance data or reflect biased assumptions
- Instruct students to repeat the exercise with country-specific parameters (e.g., Netherlands, Japan, Hungary, United States)
- Guide analysis of how adding national or cultural context changes the AI's predictions
Discussion questions
- Do the AI predictions reflect evidence-based performance data or do they perpetuate stereotypes?
- How might AI-generated predictions function as self-fulfilling prophecies if deployed in educational assessment or placement systems?
- What responsibilities do AI developers have to identify and address demographic biases in their systems?
- How should educational institutions approach AI-generated demographic predictions and analytics?
Assessment opportunity
Students research actual performance data from educational assessment literature and compare it to AI predictions, identifying discrepancies and discussing their implications.