Manipulating Statistics Study Guide (page 2)

Lesson Summary

Figures don't lie, but liars figure.

Mark Twain, American writer and humorist (1835–1910)

Numbers are facts, so numbers are always true, right? Well, not always. Sometimes people use numerical information incorrectly, either innocently or with a motive to mislead. In this lesson, we'll explore some common ways numbers are misused, including incorrectly gathering the figures, drawing the wrong conclusion, and misrepresenting the data.

We're bombarded with facts and figures every day—numerical information about what's going on in the world, who we should vote for, what we should buy, and even what we should think. The problem is, facts and figures aren't always factual. You've probably heard the old saying, "numbers don't lie." Well, they do, or rather the people who use them do!

Numbers are manipulated all the time, whether by deliberate misuse, negligence, or plain incompetence, so that what we see, hear, and read isn't always the truth. If we rely on numbers in statistics, polls, or percentages as a basis for decisions and opinions, we could be making a serious mistake. After all, people who work with numbers and those who analyze or interpret them are people. They may be biased, less than competent, or negligent, so you have to be just as concerned with the sources and quality of numbers as you are with words.

Numbers can be misused. It all happens in one, or both, of two key areas. First, numbers must be gathered. If they are collected incorrectly, or by someone with an agenda or bias, you need to know that. Second, numbers must be analyzed or interpreted. Again, this process can be done incorrectly, or misused by an individual or group. Once you learn what to look for in these two areas, you can evaluate the numerical data you encounter and rely on it only when it is objective and correct.

Manipulating Surveys

Authors, advertisers, businesses, and politicians rely on surveys, polls, and other statistics to make their points of view appear more credible and important. The problem is, it's just as easy to mislead with numbers as with words. Numbers must be gathered correctly so they can be trusted. Here are a few examples, including how numbers are manipulated so they can't be trusted.

1. Use an appropriate sample population that is
   • large enough—if the sample number is too low, it won't be representative of a larger population; asking just two people if they like a new ice cream flavor and finding that one person does doesn't mean that 50% of all ice cream eaters, a number in the millions, will like the flavor.
   • similar to the target population—if the target population includes ages 10–60, your sample can't be taken just from a junior high school
   • random—asking only union members about labor laws is not random; asking one hundred people whose phone numbers were picked by a computer is
2. Remain un-biased. Ask objective questions and create a non-threatening, non-influencing atmosphere. Compare, "Do you think people should be allowed to own dangerous firearms if they have innocent young children at home?" to "Do you think people should be allowed to exercise their Second Amendment right to own a firearm?" Also, if the person asking the question is wearing a "Gun Control Now!" or "Gun Freedom Now!" button, his or her bias pollutes the environment and will influence the answers received.

Imagine an ad that reads, "Eighty percent of respondents in a recent survey liked Smilebright toothpaste better than Brightsmile." The high percentage is meant to convince readers that most people prefer Smilebright, so you will, too. But how was that percentage figured? The survey consisted of asking five people, who already said they preferred a gel-type toothpaste, whether they liked Smilebright or Brightsmile. There was no random sampling—everyone had the same preference, which is probably not true for a larger population.

Statistics

Statistics is simply a mathematical science that gathers information about a population so that population may be described usefully. Statistics are often used to draw conclusions and make decisions based on that information. So, what's the problem?

Statistics are complicated and their problems can be numerous. In general, though, problems with statistics are similar to those of other types of numerical data; namely, they can be gathered, analyzed, and/or interpreted incorrectly, or mishandled by someone with a bias. Let's look at two common problems with statistics.

• Is it meaningful? Many parents worry when they hear that the average baby walks at 13 months. They may conclude that something must be wrong with their 18-month-old who is still crawling. But, studies have proven that at age two, there are no developmental differences between early walkers and babies who walk later. So the statistic is not meaningful; there is nothing abnormal about an 18-month-old who still crawls. Here is another example. When national standardized test scores were analyzed, it was concluded that since students from wealthy communities had higher scores, they were smarter than students in poorer communities. Is that a meaningful, accurate conclusion? Probably not; it doesn't take into account the many other variables that can explain lower test scores, such as teacher quality, less test preparation, fatigue, and even breakfast on the day of testing.

Tip

Always check the source of data. Is the source credible? Look for another source that gives similar data to strengthen the credibility.

Tip

Never trust numbers alone to prove a point. Ask yourself, "Why should I believe these statistics?" Always look for some other kind of evidence or an example to reinforce the figures.

In Short

It is just as easy to deceive with numbers as it is with words. Surveys, studies, and statistics are conducted and interpreted by researchers who might have a bias, or simply lack the skills necessary to do their jobs properly. Therefore, it is important to evaluate numbers before accepting them as truth. Ask questions about how the information was gathered, what its margin of error is, and how meaningful it is. Does the conclusion make sense, or does it seem to distort the findings? Thinking critically about the many numbers you encounter will help you to rely only on information that is objective and accurate.

Skill Building Until Next Time

• Watch a news broadcast and listen for the results of a survey or poll. Does the newscaster tell the margin of error? Why is it important to know this number?
• Look for a print or article that includes a statistic. Why was it included? Does it seem accurate and objective? How else could the advertiser or writer have made the point without using numbers?

Exercises for this concept can be found at Manipulating Statistics Practice Exercises.