15-18. Textbook questions 2.50 and 2.52 (p.178), 2.61 (p.181), 2.100 (p.212)

19. Explain what is meant: "Correlations based on averages are usually too high when applied to individuals."

20. Suppose that there is a strong positive correlation between low credit rating and probability of incarceration. Does this prove that the decision by a credit-rating bureau such as TRW or Equifax to give someone a low credit rating will make that person more likely to become a criminal? Explain your answer.

21. In the 1970s, the Federal Highway Administration solicited public comment on a proposed plan to install metric road signs on all Interstate highways. (Mile markers and distances would be phased out and replaced with kilometer information.) However, the agency bowed to public pressure and shelved its plan after receiving several thousand angry letters from citizens. Did the government’s decision reflect the will of the public? Was it the correct political decision? Explain your answers.

22. Earlier this week, the largest insurance-discrimination judgment ever ($100 million) was awarded to plaintiffs who had accused an insurance company of race discrimination. In 8 of 15 test cases, researchers posing as prospective home buyers were denied a policy when they indicated that their previous address had been in a predominantly African American neighborhood. None of those posing as previous residents of white neighborhoods received a rejection. Was this an experiment? If so, was there a control group? What arguments could you, as an attorney, have used in an attempt to defend the insurance company against these charges? [Note: There is additional evidence suggesting that race discrimination indeed occurred. However, the insurance company denies this and plans to appeal.]

23. Starting at line 130 in Table B, describe how you would simulate the sampling distribution of a public-opinion poll (samples of size 20) in which the true population parameter is 70% approval. How large would your samples have to be in order to cut the variability of the sampling distribution in half?

20. It is doubtful, though not impossible, that a rating bureau’s action of assigning a low credit rating would cause an increased probability of incarceration. While a low credit rating could conceivably put someone on the track toward ruin, and perhaps even focus additional scrutiny by the police, it is more plausible that a low credit rating and an increased risk of incarceration are common responses to living a disadvantaged life. In other words, poverty is probably a lurking variable: people who are born into poverty will tend to have low credit ratings and be more likely (for whatever reason, possibly including police discrimination) to be incarcerated. The credit rating does not cause incarceration any more than receiving a high score on the SAT causes a student to do well in a statistics class.

Key ideas: lurking variables, common response, questionable case for causation

21. While the government’s decision may in fact have reflected the will of the public, the angry responses were from an unscientific "voluntary response" survey. The responses were probably quite biased, since people willing to go along with metric conversion would be unlikely to feel strongly enough about their complacency to write a letter. Nevertheless, Washington relies heavily on the "squeaky wheel" theory, and the right of citizens to petition and complain is enshrined in our heritage. Hardly any bureaucrat—or, even less so, an elected official—could ignore thousands of impassioned letters, especially in the days before e-mail and computer-generated spam. The decision was probably politically correct.

Key ideas: bias, voluntary response, recognition of the power of a small group willing to take a stand

22. Although this was an experiment that included a control group of "white" applicants, the very small sample size and large number of potential lurking variables make the conclusion subject to, at minimum, careful scrutiny. Even if the experiment had been free from bias, we know from our classroom experiment with coin flipping that the sampling distribution with 15 trials will have considerable variability—perhaps even enough to suggest that the 8 out of 15 failures were a coincidence.

Key ideas: small sample size, possible bias, lurking variables, high variability in sampling distribution

Here are some additional questions that a lawyer might use in ruthlessly cross-examining the researchers:

Q. Did you know as you were filling out the insurance application whether your fictional previous address would be "white"? If so, what steps did you take to avoid the bias that we nearly always see in non-blind experiments?

Q. Did you consider any of the potential lurking variables (credit ratings, increase in house value, involvement of other professionals such as real estate agents, title status of previous dwelling, zoning status of previous dwelling, manner in which the forms were filled out, etc., etc.)? Did you attempt to control these variables by, for example, employing a block design? [Of course the answer is no. There weren’t enough trials to even begin to make blocks.]

Q. Was the selection of previous address (white or African American) randomized? If not, how can you assert that your experiment was free from selection bias?

Q. Did you replicate your results on enough test cases to provide high confidence that your results were statistically significant and not merely the result of chance variation? [Again the answer is no.]

Q. As I’m sure you know [voice dripping with sarcasm], the three principles of statistical design of experiments are control, randomization, and replication. Since you did not employ all three of these, precisely how did you plan to go about making your study statistically valid? Were you using other principles that haven’t yet shown up in the statistics textbooks? Do you have some statistical principles of your own that you’d like to explain to the court?

Note: Remember, there is other evidence suggesting that the company did in fact discriminate, and the jury agreed. The inflammatory questions above merely demonstrate how easy it is to criticize someone else’s experimental design. It’s much easier to criticize an experiment than to design a good one.

23. Let digits 0-6 represent "yes" (approval) and let 7-9 represent "no." Starting at line 130, look at blocks of 20 digits at a time and make a stemplot of the proportion of "yes" values. The first few entries recorded would be 12/20 (i.e., 0.6), 13/20 (i.e., 0.65), and 15/20 (i.e., 0.75). Eventually, the stemplot would tend to become bell-shaped (normal), centered on 0.70 (unbiased, but with fairly high variability). By quadrupling the sample size (i.e., looking at 80 digits at a time), we could cut the variability in half.