PARADOXES OF PROBABILITY
A treatise on probability theory wouldn't be complete without a number
of paradoxes to thoroughly confuse you. Here are a few classic ones:
Monty Hall hosted a game show called Let's Make a Deal. He presented
his game show contestant with three doors numbered 1, 2, and 3. Behind one
of the doors is the grand prize. The contestant chooses a door. Then Monty,
who knows what's behind each door, opens up one of the two remaining doors
which doesn't have the grand prize behind it. Monty then asks the contestant, "Do
you want to stay with your original choice, or would you like to switch to the
other remaining door?"
Should the contestant stay with her original choice, should she change
to the other door, or does it not make any difference?
A doctor has two drugs, A and B, which he can prescribe to patients
with a certain illness. The drugs have been rated in terms of their
effectiveness on a scale of 1 to 6, with 1 being the least effective and
6 being the most effective. Studies show that drug A is uniformly
effective at a value of 3. Drug B varies in its effectiveness. 54% of
the time it scores a value of 1, and 46% of the time it scores a value
of 5.
The doctor, wishing to provide his patients with the best possible
care, asks his statistician friend which drug has the highest
probability of being the most effective. The statistician says, "It is
clear that drug A is the most effective drug 54% of the time. Thus drug
A is your best bet."
Later a new drug C becomes available. Studies show that on the
scale of 1 to 6, 22% of the time this drug scores a 6, 22% of the time
it scores a 4, and 56% of the time it scores a 2.
The doctor, again wishing to provide his patients with the best
possible care, goes back to his statistician friend and asks him which
drug has the highest probability of being the most effective. The
statistician says, "Well, seeing as there's this new drug C on the
market, your best bet is now drug B, and drug A is your worst bet."
Show that the statician is right.
(The problem of the three chests)
Three cards are in a hat. One card is white on both sides; the second
is white on one side and red on the other; the third is red on both
sides. The dealer shuffles the cards, takes one out and places it flat
on the table. The side showing is red. The dealer now says,
"Obviously this is not the white-white card. It must be either the
red-white card or the red-red card. I will bet even money that the
other side is red." Is this a fair bet?
(Assume equal probability of a child being a boy or girl)
Question 1:
| A mother has two children. The younger one is a daughter named
Mary. What is the probability that the other child is a girl? |
Question 2:
| A mother has two children. The older one is a daughter named
Mary. What is the probability that the other child is a girl? |
Question 3:
| A mother has two children. One of them is a daughter. What
is the probability that the other child is a girl? |
Question 4:
| A mother has two children. One of them is a daughter named
Mary. What is the probability that the other child is a girl? |
In college I came across some statistics which said the accident rate
per mile was less for airplanes than it was for automobiles. Thus as
we all know traveling by air is safer than traveling by car. On the
same page however were statistics indicating the accident rate per
HOUR was MORE for airplanes than it was for automobiles. Thus it
would appear automobiles are actually safer by the hour while air
travel is safer by the mile. The reader is left to ponder the
interpretation of statistics.
HOW TO MAKE 44 A MAJORITY OF 140
“The persons who composed the Assembly of the Notables were all nominated by the
King, and consisted of one hundred and forty members. But as M. Calonne could not
depend upon a majority of this Assembly in his favor, he very ingeniously arranged them
in such a manner as to make forty-four a majority of one hundred and forty; to
effect this he disposed of them into seven separate committees, of twenty members each.
Every general question was to be decided, not by a majority of persons, but by a majority
of committees; and as eleven votes would make a majority in a committee, and four
committees a majority of seven, M. Calonne had good reason to conclude that as forty-four
would determine any general question he could not be outvoted.”
—Thomas Paine The Rights of Man Part the First. (1791)
I leave the reader with the following book recommendation:
How to Lie With Statistics
by Darrell Huff, Irving Geis (Illustrator) (1954)
A classic — still in print.
There are about 700,000 physicians in the United
States. The U.S. Institute of Medicine estimates that each year between
44,000 and 98,000 people die as a result of medical errors.
1 This makes
for a yearly accidental death rate per doctor of between 0.063 and
0.14. In other words, up to one in seven doctors will kill a patient
each year by mistake. Take gun owners in contrast. There are 80,000,000
gun owners in the United States. Yet their errors lead to "only" 1,500
accidental gun deaths per year. This means that the accidental death
rate, caused by gun-owner error, is 0.000019 per gun owner per year.
Only about 1 in 53,000 gun owners will kill somebody by mistake.
Doctors then, are 7,500 times more likely to kill somebody by mistake.
While not everybody has a gun, almost everybody has a doctor (or
several doctors), and is thus severely exposed to the human error
problem.2
It is also true statistically that one is more likely to be killed by a
family member than by a serial killer.
Should we abandon our family and move in with a serial killer?
I leave the reader to once again ponder the meaning and interpretation
of statistics.
Probability Theory is never intuitive. Our brains did not evolve to intuitively
understand probability theory correctly. The results are often surprising
and counterintuitive.
The contestant should always switch doors. Without switching doors she
has a 1/3 chance of winning. Switching doors gives her a 2/3 chance of
winning.
Another way to think about it is start with 100 doors instead of 3.
The contestant chooses a door. Then Monty opens 98 other doors revealing no
prize behind them, leaving just two doors left: the one you initially
chose, and one other. Now should the contestant switch doors?
A proper use of Bayes' Theorem is required to mathematically solve this
problem. Bayes' Theorem tells us how to properly update the odds after we are
given new information. The result is often counterintuitive.
However, before we jump to the math, let's first just reason our way through
this. (Reasoning, though, usually works only after one has determined the
correct answer by doing the math.)
Let's say you pick door #3. Think of the doors as now being divided
into two sets:
Your door {3} has a 1/3 chance of being the grand prize. The remaining doors,
{1 & 2}, taken together as a group, collectively have a 2/3 chance of having
the grand prize behind one of them.
Monty, who knows what's behind each door, now opens up one of the first two
doors which doesn't have the grand prize behind it.
Nothing has changed. Door {3} still has a 1/3 chance of being the grand prize,
and the first two doors, {1 & 2}, taken together as a group, still have a 2/3
chance of having the grand prize behind one of them. Except now there's only
one door left unopened in that group.
A computer simulation of the game using a random number
generator is good way to cross check the result.
(See http://macchiato.com/humor/monty_hall_skeptics.htm
for Java code that executes the Monty Hall problem.)
And here's the math, in case anyone cares. Again, a proper use of Bayes'
Theorem is required to mathematically solve this problem. Bayes' Theorem
tells us how to properly update the odds after we are given new information,
the new information being: The grand prize is not behind door number
<whatever door Monty opens>
For this analysis let's label the doors {A,B,C}.
The a priori probability that the prize is behind any door X, P(X) = 1/3
Let's examine the case where the contestant selects door A, and Monty
then opens door B.
The probability that Monty Hall opens door B if the prize were behind A,
P(Monty opens B|A) = 1/2
The probability that Monty Hall opens door B if the prize were behind B,
P(Monty opens B|B) = 0
The probability that Monty Hall opens door B if the prize were behind C,
P(Monty opens B|C) = 1
(Note: for the following M.o. = Monty opens)
The probability that Monty Hall opens door B is then
p(Monty opens B) = p(A)*p(M.o. B|A) + p(B)*p(M.o. B|B) + p(C)*p(M.o. B|C)
= 1/6 + 0 + 1/3 = 1/2
Then, by Bayes' Theorem,
P(A|Monty opens B) = p(A)*p(Monty opens B|A)/p(Monty opens B)
= (1/6)/(1/2)
= 1/3
and
P(C|Monty opens B) = p(C)*p(Monty opens B|C)/p(Monty opens B)
= (1/3)/(1/2)
= 2/3
In other words, the probability that the prize is behind door C is higher
when Monty opens door B, and you SHOULD switch!
Proof:
|
|
| Pr(A>B and A>C) | = | Pr(A>B | and | A>C | | | A=3 | ) | * | PR(A=3) |
| = | .54 | * | .56 | | * | 1 | |
| = | .3024 | |
| Thus A is the most effective drug ~ 30% of the time. |
| Pr(B>A and B>C) | = | | Pr(B>A | and | B>C | | | B=1 | ) | * | Pr(B=1) |
| + | Pr(B>A | and | B>C | | | B=5 | ) | * | Pr(B=5) |
|
| = | | 0 | * | .56 | | * | .54 |
| + | 1 | * | .78 | | * | .46 |
| = | | .3588 | |
| Thus B is the most effective drug ~ 36% of the time. |
| Pr(C>A | and | C>B | ) | = | | Pr(C>A | * | C>B | | | C=2 | ) | * | Pr(C=2) |
| + | Pr(C>A | * | C>B | | | C=4 | ) | * | Pr(C=4) |
| + | Pr(C>A | * | C>B | | | C=6 | ) | * | Pr(C=6) |
| = | .3388 | |
| Thus C is the most effective drug ~ 34% of the time. |
Comparing the results:
| A is the most effective drug | ~ 30% | of the time. |
| B is the most effective drug | ~ 36% | of the time. Most effective most of the time. |
| C is the most effective drug | ~ 34% | of the time. |
| = 100% | |
Granted this is a silly way of rating medications, but Simpson's Paradox
actually turned up in a (now not so) recent investigation to see if
there was sex bias in the admissions of men and women to graduate
studies at the University of California at Berkley. Independent studies
of admissions of men and women in the fall of 1973 showed a positive sex
bias against female applicants. Then when the data for men and women
were combined, there was a small but statistically significant bias in
FAVOR of women. (See "Sex Bias in Graduate Admissions: Data from
Berkeley," by P. J. Bickel, E. A. Hammel and J. W. O'Connell in
'Science', Vol. 187, February 7, 1975, pages 398-404).
The chance that the underside is red is 2 to 1.
| Answer 1: | | 1/2 | | The possibilities are Daughter-Girl, Daughter-Boy. |
|
| Answer 2: | | 1/2 | | The possibilities are Girl-Daughter, Boy-Daughter. |
|
Originally I listed the answer to question 3 as 1/3, with the possibilities being
Girl-Boy, Boy-Girl, Girl-Girl. I even added this Baysian Analysis proving the
answer was 1/3.
However, I am now convinced that, given the way the problem was worded, this answer is wrong.
Consider the following questions:
- Two coins are flipped. A piece of paper is placed over each coin so we can’t see how they landed.
I remove the paper covering one of the coins to discover that it landed Tails.
What is the probability the other coin landed Tails?
1/2
- Two coins are flipped. A piece of paper is placed over each coin so we
can’t see how they landed. I have a magic cube that will glow pink when
placed in the vicinity of a coin that landed Tails. I place the magic cube
next to the two coins and it glows pink.
What is the probability the other coin landed Tails?
The question makes no sense, because there is no ‘other’ coin.
- A mother has two children. The mother does not have two boys. What is the probability the mother has 2 daughters?
1/3
- A mother has two children. The mother does not have two boys. What is the probability the other child is a daughter?
There is no other child.
- A mother has two children. At least one child is named Mary. What is the probability the other child...
There is no other child.
- A mother has two children. One of the children is named Mary. What is the probability the other child...
We have singled out the ‘other’ child.
- A mother has two children. One of the children... What is the probability the other child...
We have singled out one of the children and can now talk about the ‘other’ child.
Finally we can revisit the original question and argue that the correct answer is 1/2, not 1/3:
“A mother has two children. One of them is a daughter. What
is the probability that the other child is a girl?”
Because of the way the question is worded, the answer must be 1/2, not 1/3.
The answer to question 4 is therefore also 1/2. Knowing the child's name tells us nothing more about the other child.
(Similarly, if we were told the child was born on a Tuesday, that also would tell us nothing more about the other child.)
Thanks to Zach Star for revitalizing my interest in this question. Zach posted 2 videos about this paradox:
