Education.com
Try
Brainzy
Try
Plus

Conducting a Study and Statistics Planning Study Guide (page 3)

By
Updated on Oct 5, 2011

Types of Studies

In determining what types of conclusions can be made from a study, two primary considerations are (1) how the units are selected for inclusion in the study and (2) how the treatments are assigned to the units.

If the study units are selected at random from a population, then inference can be made to the population from which the units were drawn. Inference can only be drawn to the units in the study if units were not randomly selected from some population. If a researcher gets volunteers to participate in a study, then conclusions can be made only for those volunteers. Often, an effort is made to argue that the units in the study are representative of some larger population. However, if someone disagrees with the results and claims that the units in the study are different and that this affected the outcome, then there is no statistical foundation upon which we could argue otherwise.

In some studies, treatments can be assigned at random to units. For example, the treatments could be a new type and a standard type of dog food. Half of the dogs available for the study could be randomly assigned to the new type of dog food; the other half would get the standard type. If the assignment is made at random, then the study is called an experiment, and thus, a cause-and-effect relationship can be claimed. Returning to the dog food study, if the dogs on the new type of dog food had improved health compared to those on the standard dog food, we could conclude that the type of food caused the difference. (More advanced methodology may be used to derive casual relationships, but they are beyond the scope of this book.)

If treatments are not assigned at random to the study units, then we can discuss associations but not cause and effect. For a long time, it was observed that people who smoked were more likely to develop lung cancer than those who did not smoke. However, smoking is not a treatment that can be randomly assigned (at least ethically) to people. Therefore, it could not be claimed that smoking caused cancer, only that the two were associated with each other.

If treatments are assigned at random to units that were randomly selected from a population of interest, then the study is an experiment with a broad scope of inference. The term broad scope of inference means that inference can be drawn beyond the study units to the whole population.

If treatments are assigned at random but the units were not randomly selected from some population, then the study is an experiment with a narrow scope of inference. Because no random selection of the study units occurred, inference can be made only to the units in the study, and this is called a narrow scope of inference.

If treatments are not assigned at random but units are randomly selected from a population of interest, then the study is a sample survey. Notice that when conducting a survey, it is not possible to assign certain treatments at random. Gender, age, and dominant hand are only three examples. Associations, but not cause-and-effect conclusions, can be concluded for the population.

If treatments are not assigned at random and units are not randomly selected from a population of interest, then the study is an observational study. Here, associations can be drawn, but only for units in the study.

The discussion in the previous paragraphs is summarized in Table 2.1. To illustrate using Table 2.1, consider the following study. Suppose the goal is to determine whether the nicotine patch increases the proportion of heavy smokers (those smoking at least a pack a day) who are able to stop smoking. The study could be conducted in several ways. First, suppose that an advertisement is placed in a newspaper asking heavy smokers who want to quit smoking to participate in a study. All interested participants are screened to be sure that they are heavy smokers and to confirm a genuine interest in quitting. Half of these are randomly assigned to wear a nicotine patch; the other half are given a patch that has no nicotine. Every participant wears a patch for six weeks. Two months after the patch is removed (eight weeks after the start of the study), each study participant is assessed to determine whether or not he or she is smoking. Because study participants are volunteers and all volunteers meeting the study criteria were included, there was no random selection of units (people) for inclusion in the study. This would correspond to the second row in the body of Table 2.1.

Table 2.1 Treatment and selection

Because there was no random selection of units, inference can be made only to the people included in the study. In practice, the argument is often made that the study participants are no different from other heavy smokers, and conclusions are made more broadly. However, if someone claims that these study participants are not representative of the population of heavy smokers and thus the conclusions do not apply to the whole population, there is no solid foundation by which to refute the claim.

The treatments (nicotine patch and no nicotine patch) were assigned at random. This corresponds to the second column in the table, so cause-and-effect conclusions can be made. That is, if the proportion of study participants who stopped smoking is significantly greater for those who wore the nicotine patch than the proportion of those who did not wear the nicotine patch, we conclude that the difference is due to the nicotine patch. The patch without nicotine is called a placebo patch because it has no active ingredients. Often, people who receive a treatment respond whether or not the treatment has any active ingredient. To be sure that a treatment, such as a patch or a pill, is effective, a treatment that appears the same but has no active ingredient is also given. The patch or pill or other item with no active ingredient is called a placebo.

In summary, the nicotine patch study is an experiment with a narrow scope of inference. It is an experiment because treatments are assigned at random. The scope of inference is narrow because people were not randomly selected from the population of heavy smokers for inclusion in the study, and thus, inference can be made only to the people in the study.

The nicotine patch study was a blinded one because the study participants did not know whether or not they had a patch with the active ingredient. A double-blinded study is one in which neither the study participant nor the individual determining the value of the response variable knows which treatment each person has received. By blinding, any tendency to favor one treatment over the other can be eliminated.

Conducting a Study and Statistics Planning In Short

In this lesson, we have discussed the key steps in conducting a study. Every decision made during a study has an impact on the analysis and interpretation of the results. The strongest conclusions are from experiments where treatments have been applied at random, allowing cause-and-effect conclusions to be made. Otherwise, we are limited to discussing observed associations between variables. By drawing the study units at random from a population, we are able to draw inference beyond the units used in the study to the population from which the units were drawn.

Find practice problems and solutions for these concepts at Conducting a Study and Statistics Planning Practice Questions.

View Full Article
Add your own comment