Conducting a Study and Statistics Planning Study Guide (page 2)
Introduction to Conducting a Study and Statistics Planning
Statistics involves the collection and analysis of data. Both tasks are critical. If data are not collected in a sensible manner, no amount of sophisticated analysis will compensate. Similarly, improper analyses can result in improper conclusions from even the best data. A key to a successful study is to establish a solid framework. In this lesson, we will outline such a framework and discuss the types of inference that can be made from different types of studies.
Steps in Planning and Conducting Studies
Most studies are undertaken to answer one or more questions about our world. Would drilling for oil and gas in the Arctic National Wildlife Refuge negatively affect the environment? Do laws mandating seat belt use increase the rates of their use? Is the flu vaccine safe and effective in preventing illness? These are the types of questions statisticians like to answer.
Planning and conducting a study can be outlined in five steps, each of which we will discuss briefly:
- developing the research question
- deciding what to measure and how to measure it
- collecting the data
- analyzing the data
- answering the question
Developing the Research Question
Statisticians often work in teams with other researchers. The team works together to determine the research question to be addressed in an upcoming study. To fully specify the research question, the study population should be identified and the goals of the study should be outlined. The statistician must understand the question(s) and the goals of the study if he or she is to be a full member of this team.
Deciding What to Measure and How to Measure It
Once the research question has been specified, the team must determine what information is needed to answer the research question. Identifying what variables will be measured and deciding how they will be measured is fundamental. Sometimes, this step is obvious (as in a study relating salaries of individuals to educational level). At other times, this is extremely challenging (as in a study relating attitudes toward school to intelligence).
In some studies, a comparison of two or more regimens or procedures may be the focus of the research question. As an illustration, a study could be used to determine whether students perform better on English tests if they study in a quiet environment or while listening to classical music. To answer the question, some students would study in a quiet environment; others would study while listening to classical music. The scores on the English test for each group would be used to answer the research question. The study environments (quiet or classical music) would be the treatments in this study. A treatment is a specific regimen or procedure assigned to the participants of the study.
Collecting the Data
Good data collection is a crucial component of any study. Because resources are always limited, the first question is whether an existing data source exists that could be used to answer the research question. If existing data are found, the manner in which the data were collected and the purpose for which they were collected must be carefully considered, so that any resulting limitations they would impose on the proposed study can be evaluated and judged to be acceptable. If no existing data are found, a careful plan for data collection must be prepared. The manner in which data are collected determines the appropriate statistical analyses to be conducted and the conclusions that can be drawn.
Analyzing the Data
Before data are collected, the analysis should be outlined. With the analysis and potential conclusions in mind, the research question should be reviewed to confirm that the planned study has the potential of answering the question. Too often, studies are conducted before the researchers realize they have no idea how to analyze the data or that the collected data cannot be used to answer the research question. The statistician should verify that the data collection protocol was properly followed. Each analysis should begin by summarizing the data graphically and numerically. Then the appropriate statistical analyses should be conducted.
Answering the Question
Through interpretation of the analysis results, we learn what conclusions can be drawn from the study. The aim is to answer the research question using the conclusions drawn from the study. Sometimes, we are unable to answer the question or are able to only partially answer it. At the conclusion of any study, the research team should reflect on what was learned from the study and use that to direct future research.
Selecting the Sample
Most of the inferential methods introduced in the text are based on random selection. The simplest form of random selection is simple random sampling. A simple random sample of size n is one drawn in such a manner that every possible sample of size n has an equal chance of being chosen.
It is important to realize that, if every unit in the population has an equal chance of being included in a sample, the sample may still not be a simple random one. To see this, suppose that a company has two divisions, A and B. There are 700 employees in division A and 300 in division B. The management decides to take a sample of 100 employees. To do this, they write each employee's name on a chip and put the chip in bowl A or B, depending on whether the employee is in division A or division B, respectively. The chips are thoroughly mixed in each bowl. Seventy chips are drawn from bowl A and 30 chips are drawn from bowl B, and the employees whose names are on the selected chips comprise the sample. Each employee has a 1 in 100 chance of being included in the sample; however, this is not a random sample.
Only samples with 70 division-A employees and 30 division-B employees are possible; it would not be possible to have, for instance, a sample with 50 division-A employees and 50 division-B employees. Because not all samples of size 100 are equally likely to be selected, this is not a random sample. In Lesson 14, we will discuss other methods of random selection.
Care must be taken in selecting a sample so that it is not biased. Bias is the tendency for a sample to differ in some systematic manner from the population of interest. Some common sources of bias are selection bias, measurement bias, response bias, and nonresponse bias. Selection bias occurs when a portion of the population is systematically excluded from the sample. For example, suppose a company wants to estimate the percentage of adults in a community who smoke. If a telephone poll is conducted, adults without telephones would be excluded from the sample, and selection bias would be introduced.
Measurement bias, or response bias, occurs when the method of observation tends to produce values that are consistently above or below the true value. For example, if a scale is inaccurately calibrated, observed weights could be consistently greater than true weights, resulting in a measurement bias. The way in which a survey question is worded could influence the response, leading to bias. For example, suppose that a survey question was stated as follows: "Many people think driving motorcycles is dangerous. Do you agree?" When stated in this way, the proportion of those agreeing will tend to be larger than would have been the case if the question had been phrased in a neutral way. The tendency of people to lie when asked about illegal behavior or unpopular beliefs, characteristics of the interviewer, and the organization taking the poll could be other sources of response bias.
Often in surveys, some people refuse to respond. Nonresponse bias is present if those who respond differ in important ways from those who do not participate in the survey. In a survey of gardeners, those with smaller gardens were much more likely to respond than those with large gardens. Because some of the questions were related to the size of the garden, this nonresponse resulted in response bias.
- Kindergarten Sight Words List
- First Grade Sight Words List
- 10 Fun Activities for Children with Autism
- Definitions of Social Studies
- Signs Your Child Might Have Asperger's Syndrome
- Curriculum Definition
- Theories of Learning
- Child Development Theories
- A Teacher's Guide to Differentiating Instruction
- Netiquette: Rules of Behavior on the Internet