Manipulating Statistics Study Guide (page 2)

based on 6 ratings
Updated on Sep 19, 2011


Remember, surveys can't prove anything with 100% certainty unless the sample questions 100% of the population.

Margin of Error

Most survey results end with a statement such as "there is a margin of error of three percentage points." What does this mean? It tells how confident the surveyors are that their results are correct. The lower the percentage, the greater their confidence. A 3% margin of error means that the sample population of the survey could be different from the general population by 3% in either direction. Let's say a survey concluded that "55% of Americans want to vote for members of the Supreme Court." If there is a 3% margin of error, the results could be 58%, 52%, or anywhere in between, if you conducted the identical survey asking another group of people.

Knowing the margin of error is important, especially in political polls. Imagine a headline that reads, "Smith's lead slips to 58%; Manotti gaining momentum at 37%." The accompanying article states that last week, the results were 61% to 34%, with a 4% margin of error. That means there's really no difference between the two polls. No one is "slipping" or "gaining momentum." The margin of error tells the real story; the news article is wrong.

Correlation Studies

Once numbers are gathered, they must be interpreted or evaluated, and this step affords many opportunities to distort the truth. For example, researchers often do correlation studies to find out if a link exists between two sets of data. Here are two questions someone might use for a correlation study:

  • Is there a connection between the full moon and an increase in birth rates?
  • Does having a high IQ indicate that you will have a high income level?

Imagine that research at five area hospitals shows that during a full moon, an average of 4% more babies are born than on nights with no full moon. You could then say there's a small but positive correlation between full moons and birth rates. But many studies have shown that any correlation is really due to chance. No evidence has been found to support the theory that the moon's phases affect human behavior in any way. So, even though you found a positive correlation, it doesn't necessarily mean there's a cause-and-effect relationship between the two elements in the study.

For the second question, if a study showed that Americans with the top 5% of IQ scores made an average of $22,000 a year, while those in the middle 5% made an average of $40,000, you would say there is a negative correlation between IQ and income levels. To describe the results of the study, you could say that there is no evidence that IQ determines income level. In other words, you do not need to have a high IQ to make a lot of money.

This conclusion is obvious. But let's look at how these same correlation study results can be used to come up with a ridiculous conclusion. The second example shows that there is no connection between a high IQ and a high income level. Is that the same as saying that "the dumber you are, the more money you will make?" Of course it isn't. This type of conclusion shows one of the dangers of correlation studies. Even if the study uses accurate data, the way in which it is interpreted can be wrong, and even foolish. When you encounter a correlation study, as with survey and poll results, do not assume the numbers and conclusion are correct. Ask questions, and look at supporting data. Does the study make sense? Or does it seem too convenient for the advertiser/politician/reporter/author who is using it? Think critically, and do not rely on anyone's numbers until you determine they are true and valid.

View Full Article
Add your own comment