STAtstravaganza Quiz #1—Results from 7/8/2010
1. Mr. Hansen asked each of the 34 students to select a codename from a list of 56 possibilities. The most popular codename was Platypus, chosen by 6 students. Other names chosen were Alpha (2), Delta, Glee (2), Hotel, India, Jazz (2), Mango (4), Mumble, Ratchet, Sierra, Sugar, Sweat, Swill, Tango, Tickle, Trash, Tumble, Uncle, Zing, and Zulu (3). The other 35 codenames, shown below, were unused.
Mr. Hansen pointed out that there had been a large number of “collisions” (duplications of codename choices). No explanation was offered for why Platypus was so popular.
Mr. Hansen then posed a question: If all students had been forced to choose randomly from the list of 56 codenames, what would be the probability that all the codenames would be unique? Students agreed that the answer would be about 50% or maybe a little lower. One “common sense” approach, which happens to be completely wrong, is to compute 32/56 = .5714, which is about 60%, leaving (seemingly) only about a 40% chance that everyone would have unique codenames. However, that is a completely incorrect answer, since the correct probability is 0.0002%. There are only about 2 chances in a million that all 34 students would be able to randomly choose unique codenames.
2. For the second question, students were asked to enter their birthdate. Here, with 366 days to choose from (remember, 1992 was a leap year) and only 34 students in the room, duplications would seem to be unlikely. However, two students were born on July 20. The probability of no duplications of birthdates drops below 50% as soon as there are 23 or more people in the room. With 34 people, the probability of no duplications is only about 20.6%. Therefore, Mr. Hansen was not taking much of a risk when he predicted that there would be at least one birthday duplication. (In fact, counting Mr. Hansen, there were two birthday duplications, since Matthew is exactly 31 years younger.)
3. A statistic is a number computed from data. Of the 34 students, 16 received at least partial credit. The winning answer was submitted by the student with codename Glee and a May birthdate: “A statistic is a numerical representation of various surveyed information.” That is not the textbook definition, but it is darned close.
4. This group of 5 questions was based on Dr. Hans Rosling’s TED video. His Swedish undergraduates had a mean score of 1.8 correct (confidence interval 1.4 to 2.2), and the professors had a mean score of 2.4 (C.I. 2.0 to 2.8). The “chimpanzee score” is 2.5. In our class, students answering the real questions about child mortality had a mean score of 1.9 (C.I. 1.3 to 2.5), while those answering randomly “LEFT” or “RIGHT” basically matched the chimpanzees (mean 2.4, C.I. 1.8 to 3.0).
What was the lesson of that exercise? Human beings are overconfident about what they think they know. Or, as Will Rogers supposedly said, “It isn’t what we don’t know that gives us trouble, it’s what we know that ain’t so.” Two of the largest recent disasters in America, namely the financial collapse of 2008 and the BP oil spill of 2010, both appear to have been caused by precisely such overconfidence. For example, if the BP officials in charge of sealing up the Deepwater Horizon wellhead had not confidently ordered replacing drilling mud with seawater shortly before the explosion, there is a good chance that the spill would not have occurred. It appears that they greatly underestimated the risk. If they had asked me, somebody who knows next to nothing about oil drilling, I would have been unwilling to remove the drilling mud for fear of making a mistake.
Similarly, if you had asked me in 2008, “Does it sound like a good idea to have an investment bank leveraged at 33-to-1 with bets on subprime mortgages?” I would have said, “I don’t really know enough about it to say for sure, but 33-to-1 sounds like a pretty low margin for safety if anything goes wrong.” I remember reading newspaper articles in the spring of 2008 saying that a majority of home loans were “nontraditional” (no money down, interest-only, negative amortization, no documentation of income, or other similar situations), and I remember thinking, “Hmmm, this can’t be good.” Meanwhile, the extremely well-paid experts on Wall Street, some of whom were taking home 7- and 8-figure annual bonuses and more than that in stock profits, confidently said that all of that was just fine, and that if they didn’t make those subprime mortgage deals, someone else would. I guess they were a little overconfident, no? Of course, it is the 7 or 8 million Americans who lost their jobs who had to pay the price. The Wall Street experts, for the most part, are still earning good incomes. Many of them are still employed at multimillion-dollar salaries, because there is no one else who “knows enough” to clean up the mess.
5. What is the unfunded future liability of Medicare and Medicaid? This refers to the amount over and above what you, your children, and your grandchildren will pay in taxes (heavy taxes) to fund these programs. The unfunded liability is approximately $100 trillion, or more than $300,000 per person in the United States. That’s right: You are already in debt to future generations of Americans by more than a quarter of a million dollars, just for Medicare and Medicaid. The Social Security unfunded liability, and even the current national debt, are both small by comparison, adding “only” another $60,000 or so per person.
Only 4 students out of 34 chose the correct answer to this question. Again, that is lower than the result that would be obtained by random guessing.
Do members of Congress know this? Many of them are aware, of course, that something needs to be done to control medical costs. However, nobody has managed to accomplish much so far. The main reason is that this problem, which threatens to destroy our country, is still farther out in the future than the next election cycle. Therefore, nobody has the courage (or political cover) to tackle the problem head-on.
Maybe one of you can do something! That is my great hope. At a minimum, maybe you can tell someone in the U.S. House or Senate that each citizen already owes more than $300,000 to the people of the future. I’m not sure the words “billion” and “trillion” have any real meaning in Washington anymore.
6. Students were exposed to a 30-second musical clip (an excerpt from Samuel Barber’s violin concerto, for anyone who might be curious) and asked to estimate its length. Answers ranged from 10 seconds to 78 seconds, with a mean of 29.1. Well done! (On average, at least.)
Scoring for this question was 1 point for each student who guessed within 25% of the true value. More than half the class, 18 students, managed to do that. The distribution of guesses was approximately bell-shaped (normal), with the exception of 3 students who guessed 60 seconds or more.
7. The final question was designed to test metaknowledge (how well you know what you know). Students were asked to estimate how many of the previous questions they had answered correctly: #1 and #2 did not count, #4 had 5 parts, and questions #3, #5, and #6 counted for 1 point each. That meant that anywhere from 0 to 8 questions could have been answered correctly. Bonus points for the accuracy of the score estimate were then awarded as follows:
|
Error in score estimate |
Bonus points |
|
>2 points |
0 |
|
from 1 to 2 points, inclusive |
1 |
|
<1 point |
2 |
Students did fairly well on the metaknowledge question, with 20 out of 34 earning at least 1 bonus point. There were 2 students who left #7 blank. Of the remaining 12 students, 8 were overconfident and 4 were underconfident. Once again, we saw evidence of a bias toward overconfidence.
Final Standings
Here are the overall results, ranked by final score:
|
Codename |
Birth month |
Question #4 |
Raw score |
Estimate (#7) |
Bonus |
Final |
|
Sugar |
June |
countries |
5 |
5 |
2 |
7 |
|
Platypus |
December |
L/R |
5 |
6 |
1 |
6 |
|
Zulu |
March |
countries |
4.75 |
6 |
1 |
5.75 |
|
Alpha |
February |
countries |
3.5 |
3 |
2 |
5.5 |
|
Mango |
June |
L/R |
3.5 |
4 |
2 |
5.5 |
|
Platypus |
Aug. (2nd half) |
countries |
3.5 |
3 |
2 |
5.5 |
|
Uncle |
February |
countries |
3.5 |
4 |
2 |
5.5 |
|
Alpha |
October |
L/R |
5 |
2 |
0 |
5 |
|
Jazz |
May |
L/R |
4 |
3 |
1 |
5 |
|
Platypus |
May |
L/R |
4 |
5 |
1 |
5 |
|
Platypus |
June |
countries |
4 |
5 |
1 |
5 |
|
Platypus |
Aug. (1st half) |
L/R |
4.75 |
2 |
0 |
4.75 |
|
India |
December |
L/R |
3.5 |
5 |
1 |
4.5 |
|
Jazz |
July |
countries |
4.5 |
2 |
0 |
4.5 |
|
Mango |
May |
countries |
3.5 |
2 |
1 |
4.5 |
|
Glee |
September |
countries |
2.25 |
2 |
2 |
4.25 |
|
Mumble |
September |
countries |
2.25 |
3 |
2 |
4.25 |
|
Tumble |
January |
L/R |
3.25 |
5 |
1 |
4.25 |
|
Hotel |
January |
L/R |
3 |
2 |
1 |
4 |
|
Mango |
March |
countries |
3 |
2 |
1 |
4 |
|
Zing |
August |
L/R |
3 |
2 |
1 |
4 |
|
Zulu |
November |
L/R |
4 |
8 |
0 |
4 |
|
Sweat |
February |
countries |
2.25 |
4 |
1 |
3.25 |
|
Zulu |
May |
L/R |
3.25 |
6 |
0 |
3.25 |
|
Delta |
October |
L/R |
3 |
0 |
0 |
3 |
|
Platypus |
March |
countries |
3 |
6 |
0 |
3 |
|
Glee |
May |
L/R |
2 |
5 |
0 |
2 |
|
Trash |
July |
L/R |
2 |
no answer |
0 |
2 |
|
Tango |
February |
countries |
1.25 |
6 |
0 |
1.25 |
|
Mango |
July |
countries |
1 |
no answer |
0 |
1 |
|
Ratchet |
May |
countries |
1 |
5 |
0 |
1 |
|
Swill |
October |
countries |
0 |
2 |
1 |
1 |
|
Tickle |
June |
L/R |
1 |
4 |
0 |
1 |
|
Sierra |
September |
countries |
0 |
6 |
0 |
0 |
|
|
|
|
|
|
|
|
|
|
|
Mean |
3.01 |
3.91 |
0.79 |
3.81 |
|
|
|
Std. Dev. |
1.38 |
1.82 |
0.77 |
1.74 |
|
|
|
|
|
|
|
|
|
|
|
Min. |
0 |
0 |
0 |
0 |
|
|
|
25th %ile |
2.25 |
2 |
0 |
3 |
|
|
|
Median |
3.25 |
4 |
1 |
4.25 |
|
|
|
75th %ile |
4 |
5 |
1 |
5 |
|
|
|
Max. |
5 |
8 |
2 |
7 |
Other Lessons
For what it’s worth, May is the most popular month for SPS birthdays, and November is the least popular. The average age of SPS students attending the statistics presentation was 17.4 years, with a standard deviation of 0.4 years.
Another lesson from the presentation was that astonishing coincidences are fairly common. For example, with 56 codenames to choose from, two students with August birthdays chose the codename Platypus!
Finally, an unusual aspect of human nature, mentioned in several of the books listed below, is that people become more confident as they have more information, even if the information is of little or no statistical value. For example, studies on the ability of professional gamblers to predict the winners of horse races showed that with only a few pieces of information, they could make fairly accurate predictions. However, the quality of their predictions did not improve substantially when they were given more information. What did increase, dramatically, was the confidence that they attached to their predictions. Sadly, as we have seen recently, such confidence can be misguided and can lead to disastrous consequences.
We saw an inkling of this phenomenon (i.e., overconfidence increasing as information increases) in our class, although the sample sizes were too small for the results to be statistically significant. (What I mean by that is that the differences, although interesting, were small enough that chance alone could account for them.) Students who were given random “LEFT/RIGHT” choices for question #4 were overconfident in their score predictions by an average of 0.3 points, whereas students who were given actual names of countries, and who probably thought they had more of a factual basis for making an estimate, were overconfident by an average of 1.0!
This is really scary. What it suggests is that the longer a person is an “expert,” the longer he or she reads up on a subject, the longer other people feed information in the direction of the “expert,” and the more respected a person is, human nature creates a strong bias for more and more overconfidence. Humans are overconfident to begin with, and people with lots of information at their fingertips tend to be even more overconfident. Yikes! The only antidote to this problem is to cross-check oneself constantly and to take a daily dose of humility.
The technical term for humility is calibration. Humans, as a species, are very poorly calibrated, and that is probably a good thing in terms of survival through the centuries. After all, if you were a caveman, you probably wouldn’t want to stick around until the next spring if you knew, statistically, that you were likely to be chewed up by a saber-toothed tiger or to be stepped on by a mastodon. A little unrealistic optimism could really be helpful in the middle of a cold, dark winter! And so, we plunge forward, dramatically overconfident, even though overconfidence may cause huge catastrophes in the modern world.
Recommended Reading
There are many recent books on the subject of statistics that you may actually enjoy reading. None of these require any skill with numbers or calculators, merely an ability to enjoy unexpected connections in data. Each link connects to Amazon.com, where you can either purchase the book or note the ISBN so that you can buy it elsewhere. All of these books are widely available in public libraries as well.
Black Swan, The by N. N. Taleb
Freakonomics by S. Levitt and S. Dubner
Outliers by M. Gladwell
Predictably Irrational by D. Ariely
Super Crunchers by I. Ayres
SuperFreakonomics by S. Levitt and S. Dubner (Warning: R-rated. Be sure to get your parents’ permission before reading this.)
Blank copies (in case you
want to give the quiz to your friends)
STAtstravaganza Quiz #1
1. Select a codename for yourself from the choices below. (Circle your answer.)
|
|
Alpha |
Delta |
Gang |
Jazz |
Mumble |
Seem |
Tango |
Uncle |
|
|
Bingo |
Donkey |
Glee |
Jumble |
Nose |
Sierra |
Think |
Uniform |
|
|
Brain |
Drain |
Glue |
Key |
Platypus |
Stubble |
Tickle |
Warthog |
|
|
Bravo |
Drum |
Glyph |
Kilowatt |
Ramble |
Stumble |
Topaz |
Whistle |
|
|
Cackle |
Echo |
Golf |
Kite |
Ratchet |
Sugar |
Trash |
Yankee |
|
|
Clang |
Feel |
Hotel |
Log |
Right |
Sweat |
Trim |
Zing |
|
|
Crumb |
Fox |
India |
Mango |
Rotten |
Swill |
Tumble |
Zulu |
2. What is your birthdate (MM/DD/YYYY)? _____ / _____ / ___________
3. What is a statistic? Please give a definition. (Not an example, please, but a definition).
________________________________________________________________
________________________________________________________________
4. Which country has the higher infant mortality rate of each pair? Please circle your answers.
(a) Sri Lanka vs. Turkey
(b) Poland vs. South Korea
(c) Malaysia vs. Russia
(d) Pakistan vs. Vietnam
(e) Thailand vs. South Africa
Remember, for each pair of countries, circle the one where the INFANT DEATH RATE per thousand live human births is higher.
5. What is the unfunded future liability of Medicare and Medicaid? Circle your answer:
(A) less than $100 billion
(B) $100 billion to $500 billion
(C) $500 billion to $50 trillion
(D) $50 trillion to $500 trillion
(E) more than $500 trillion
6. Approximately how long (in seconds) did the musical clip last? ______________
7. Questions 1 and 2 are not scored. However, that leaves 8 other questions (3, 4a-4e, 5, and 6) that could be scored. How many of those do you think you got correct? Circle your answer.
0 correct 1 correct 2 3 4 5 6 7 8
STAtstravaganza Quiz #1
1. Select a codename for yourself from the choices below. (Circle your answer.)
|
|
Alpha |
Delta |
Gang |
Jazz |
Mumble |
Seem |
Tango |
Uncle |
|
|
Bingo |
Donkey |
Glee |
Jumble |
Nose |
Sierra |
Think |
Uniform |
|
|
Brain |
Drain |
Glue |
Key |
Platypus |
Stubble |
Tickle |
Warthog |
|
|
Bravo |
Drum |
Glyph |
Kilowatt |
Ramble |
Stumble |
Topaz |
Whistle |
|
|
Cackle |
Echo |
Golf |
Kite |
Ratchet |
Sugar |
Trash |
Yankee |
|
|
Clang |
Feel |
Hotel |
Log |
Right |
Sweat |
Trim |
Zing |
|
|
Crumb |
Fox |
India |
Mango |
Rotten |
Swill |
Tumble |
Zulu |
2. What is your birthdate (MM/DD/YYYY)? _____ / _____ / ___________
3. What is a statistic? Please give a definition. (Not an example, please, but a definition).
________________________________________________________________
________________________________________________________________
4. Randomly choose a direction (LEFT or RIGHT) for each question. Only one is correct, and your job is to try to guess correctly. Please circle your answers.
(a) LEFT vs. RIGHT
(b) LEFT vs. RIGHT
(c) LEFT vs. RIGHT
(d) LEFT vs. RIGHT
(e) LEFT vs. RIGHT
Remember, for each pair of choices, circle the direction that you guess to be correct.
5. What is the unfunded future liability of Medicare and Medicaid? Circle your answer:
(A) less than $100 billion
(B) $100 billion to $500 billion
(C) $500 billion to $50 trillion
(D) $50 trillion to $500 trillion
(E) more than $500 trillion
6. Approximately how long (in seconds) did the musical clip last? ______________
7. Questions 1 and 2 are not scored. However, that leaves 8 other questions (3, 4a-4e, 5, and 6) that could be scored. How many of those do you think you got correct? Circle your answer.
0 correct 1 correct 2 3 4 5 6 7 8