The ticket scanner that's 95% accurate
A scanner that's 95% accurate stopped a ticket. What's the probability that it's fake?
Explore
Each dot is a person in line: 1,000 in all.
Lock in your guess in the panel to see who has a fake ticket and who got stopped.
real ticket fake ticket framed: stopped by the scanner
Show the line as a table
| Ticket | Stopped | Let in | Total |
|---|---|---|---|
| Fake | |||
| Real | |||
| Total | 1,000 |
Your guess
At tonight's concert, 20 out of every 1,000 tickets are fake. The turnstile scanner is right 95% of the time: it stops 95 of every 100 fake tickets and lets in 95 of every 100 real ones.
The scanner stopped a ticket. What's the probability that it's fake?
Count who got stopped
Each orange triangle is a person with a fake ticket. Each frame is a person the scanner stopped. Tap Group the line, below the grid, to move the stopped people to the front.
Change the concert: the fakes
The scanner is still 95% accurate, but now the concert has more fake tickets: 500 out of every 1,000 instead of 20.
Now keep Scanner accuracy at 95% and move Fakes per 1,000 up to 500, below the grid.
Change the concert: the accuracy
Set Fakes per 1,000 back to 1 and raise Scanner accuracy from 95% to 99%. Then compare with what you saw in task 3.
Challenge
At a festival, 100 out of every 1,000 tickets are fake, and the scanner is 90% accurate. A ticket was stopped. What's the probability that it's fake? Do the math and lock in your guess before you touch the controls.
If you haven't read it yet, read Why it came out so low first, below.
Lock in your guess first. Until then, the numbers for the line stay hidden.
Extra challenge
Now the scanner is 99% accurate. What's the smallest number of fakes per 1,000 that makes at least half of the stopped tickets fake?
Why it came out so low
The explanation unlocks when you finish tasks 2, 3, and 4, so it doesn't give away the answers.
In the line of 1,000 people, only 20 had a fake ticket, and the scanner stopped 19 of them (95% of 20). The other 980 tickets were real, and the scanner gets 5% of those wrong: it stopped 49 people who hadn't done anything wrong. That's 68 people stopped, and only 19 of them with a fake ticket.
The 95% answers a different question: out of the fake tickets, how many does the scanner stop? At the turnstile, the question runs the other way: out of the people stopped, how many have fake tickets? When fake tickets are rare, the scanner's few mistakes on the crowd of real tickets add up to more people than its hits on the few fakes.
A probability calculated only inside one group (here, only among the people stopped) is called a conditional probability. A test for a rare disease has the same problem: the rarer the disease, the bigger the share of positive results that come from healthy people.
For teachers
Objective
By the end, students can explain, by counting in a population of 1,000, why a scanner that's 95% accurate can be wrong about most of the people it stops when fake tickets are rare. This is the idea of conditional probability (and the base rate effect) without Bayes' formula: everything is counting people.
Prerequisites
Finding a percentage of a number, turning a ratio like "19 out of 68" into a percentage, and the idea of probability as the outcomes you care about divided by all possible outcomes. Students don't need to have seen probability trees or the multiplication rule.
How to use in class
About 25 minutes. The tasks sit in the panel next to the line, one at a time. Project the header and task 1 and ask every student for a guess, out loud or on paper (5 minutes). Write on the board how many said 95%. Lock in a guess (or tap I don't know) and let the class do task 2, counting the people stopped, first with the line scattered, then grouped (5 minutes). Compare the 28% with the guesses on the board. In pairs, the class does tasks 3 and 4 (Change the concert): they bet whether the probability goes up, goes down, or stays the same with 500 fakes before touching anything, and then say which control changed it more (5 minutes). The explanation below the simulation unlocks after tasks 2 to 4; read it with the class and assign the challenge (task 5) individually, with the math done in a notebook before locking in the guess; the numbers for the line appear only after the guess is locked in (10 minutes). The extra challenge, after task 5, is for students who finish early or for high school classes.
The starting case (20 fakes, 95%) and the challenge (100 fakes, 90%) come out in whole people. Many other settings of the controls don't: with 1 fake and 99%, the exact math gives 0.99 fake tickets stopped and 9.99 real tickets stopped. In those cases the line and the table round to whole people, the counts appear with "about," a note shows the exact math, and the probability always comes from the exact math (9%, not the 9.1% of 1 in 11). If a student notices, it's a good conversation: in a line of 100,000 people the math would come out without rounding.
Common mistakes
Answering 95% at the start: this is the expected mistake, and the activity keeps the guess for the comparison. Flipping the question, treating "out of the fakes, how many are stopped" as "out of the people stopped, how many are fake": it shows up as an answer of 90% in the challenge, and the feedback points this out. Ignoring how rare the fakes are and thinking only accuracy matters: a student with this idea says "they need a better scanner"; task 4 shows that with 1 fake in 1,000, even 99% doesn't fix it. Counting only one kind of stopped person: the count feedback says which kind was missing.
Discussion questions
- If the scanner stopped you, what should the concert staff do before pulling someone out of line?
- Why does a 95% accurate scanner look great to the company selling it and bad to someone stopped by mistake?
- At a concert where half the tickets are fake, is the 95% scanner good? Why?
- The book tells how, in World War II, the military wanted to add armor to the parts of returning planes that had the most bullet holes, and the mathematician Abraham Wald told them to armor the other parts. What "flipped" question were they asking?
- What other alarms do you know that go off by mistake much more often than for real?
Based on Contemporary Mathematics, OpenStax, section 7.9, under the CC BY-NC-SA 4.0 license. This activity is under the same license. Access the book for free at openstax.org.