Is the AI Flagging System Fair? — Using Simulation to Reason About Chance
Task
Lesson Description In this lesson, students use Tuva’s simulation environment to model a probabilistic process and evaluate whether an observed result is surprising under a stated probability claim. Students interpret simulated results to reason about chance and evidence, without calculating probabilities by hand. Learning Objectives Students will be able to: Interpret a simulation as a model of a random process Identify what simulation parameters represent in context Use simulated outcomes to judge whether a result is unusual Decide whether data provide evidence against a probability claim
Introduction A social media platform uses an AI system to flag posts that may violate its rules. According to the company, 17% of posts are incorrectly flagged. During one review session of 30 posts, a moderator finds that 9 posts were incorrectly flagged. In this lesson, you will use simulation to decide whether this result could reasonably happen by chance if the company’s claim is true.
Topics and standards: Math, Simulation, Probability & Random Variables, Statistics & AP Statistics, Assessment, AP Statistics