There are two general categories of experimental research—true experimental design and quasiexperimen-tal design (Gribbons & Herman, 1997). The word “quasi” in Latin means as if or almost. Considering this, quasiexperimental research could be described as a best attempt at an experiment when it is impossible, or not reasonable, to meet all the criteria of a true experiment. This type of research is typically identified as being void of randomization of either subjects or treatment and/or the lack of comparison groups. Yet, there is still an attempt to isolate the treatment. As an overarching goal, the body of quasiexperimental research attempts to answer questions such as: “Does a treatment or intervention have an impact?” and “What is the relationship between program practices and outcomes?” (Dimsdale and Kutner, 2004).
True experiments are ideal when it comes to being able to isolate any type of statistical relationship, with the potential to infer causality. Even so, prior to designing an experiment, it is necessary to consider a few key elements. Can the individuals and/or other elements in your study be precisely classified? Can you select random individuals or other elements in your study? Is the process of randomization fair to all involved in the study? Should the answer be “no” to any of these questions, quasiexperi-mental designs then surface. Educational research, and the techniques or interventions that are introduced, seem relatively harmless to the general eye; thus, many researchers continue to emphasize true experiments (Kid-der, 1981). It is never so simple; however, as effectiveness, efficiency, and feasibility also need to be considered. There are other ethical considerations as well. If there is any risk of harm from delivering or withholding services to someone in the sample, then an experimental design cannot be employed. For these reasons, in fields such as education, psychology, and criminal justice, quasiexper-imental research is often utilized. As a whole, the research community tends to support the use of and recognize the utility of quasiexperimental research (Campbell & Stanley, 1963; Cook & Campbell, 1979).
Quasiexperimental research and experimental research both attempt to create a design scheme in which the concluded results can be thought of as the best, most logical solution to the question at hand. Quasiexperimen-tal designs tend to do this through a comparison of existing groups. Experimental designs accomplish the same goal through random assignment of individuals to interventions or treatments (Michigan State University, College of Education, 2004).
Experiments, especially large-scale, are designed to control for the influences of extraneous variables. The goal is to allow for a maximum level of certainty regarding the impact of an intervention. Specifically, experimental designs must have random selection of subject, use of control groups, random assignments of individuals to the control and experimental groups, and random assignment of groups to the intervention (Henrichsen, Smith & Baker, 1997). The strongest comparisons are made through the ability to conduct a true experiment (Grib-bons & Herman, 1997). To this extent, quasiexperimental designs attempt to rule out unrelated explanations so that the outcome can be attributed only to the experimental intervention. The task is not as straightforward, but through efforts such as matching subjects and statistical analysis, the true experiment is mimicked (Morgan, Gliner & Harmon, 2000; SERVE Center, 2007). Specifically, trend replaces the word cause in quasiexperimental research. The goal of quasiexperimental research is to discover the one trend that is a result of the treatment or intervention. Clearly, this is not a simple or direct task and error exists in that process.
The data collection and analyses are a point of overlap and a point of distinction between experimental and quasiexperimental research. While standardized assessments are utilized in both approaches, they are the singular mechanism to the experimental design. In contrast, quasiexperimental design also utilizes such means as surveys, interviews, and observations. The statistical techniques also have the same appearance, clean and simple. In contrast, quasiexperimental research uses an array of analysis techniques including the t-test, but also extending to correlation, regression, and factor analysis (Dims-dale & Kutner, 2004).
Quasiexperimental designs are typically employed if random assignment is not practical, or even impossible. Without randomization, typical issues surface. Even with a comparison group, the concern surfaces of how alike the groups are from the onset. Also, with the loss of control in quasiexperimental designs, it is of concern whether both groups are in some manner exposed to the intervention, intentionally or not. Nonetheless, the biggest weakness of quasiexperimental designs may also indicate the greatest strength—a broader scope of the research design. The controlled, randomized design of true experiments typically lends itself to a very limited, narrow view of the topic of interest (Gribbons & Herman, 1997). In all cases, when human subjects are involved, there is never a 100% guarantee that the results of an intervention can be completely attributed to the intervention itself, with no regard for the opinions and practice of the individuals involved. Simply stated, true experiments work well in laboratory settings. Quasiexperiments work well in natural settings (Schoenfeld, 2006).
Quasiexperimental research is designed with the intent to be as much like a true experiment as possible. The two traditional platforms are: (1) matching studies, in which participants are compared with individuals that are comparable on variables of interest that do not receive the intervention; and (2) interrupted time series, in which observations made prior to an intervention are compared again and additional observations are made after the intervention has happened. These types of studies present themselves in various formats (Prater, 1983).
The general goal of quasiexperimental research is to investigate cause and effect relationships. This approach to research allows for greater understanding of program features and practices. Because there is a loss of control in the quasiexperimental design, it is necessary for the researcher to decide what and when to measure (Dawson, 1997). What follows is a sample outline of designs. An X represents the group being exposed to a treatment or intervention. An O represents an observation or measurement. Temporal order of events is designated from left to right. The dotted line in each design is an indication of the lack of random assignment of subjects to the groups (Prater, 1983).
The One-Group Posttest Design is a one-shot case study. It simply has a treatment, X, and posttest, O with no control group. This design is best implemented as an evaluation model. It should be used only when there is no available comparison group or pretest data.
In the Static-Group Comparison Design, a pre-test, X, is given to only one group, while the post-test, O, is given to both the control and experimental group. This design is comparable to the One-Group Posttest Design, with the addition of a control group for comparison purposes. No randomization is present, instead two groups are arbitrarily selected, and one is labeled as experimental and the other control.
The most common quasiexperimental design is the Nonequivalent Control-Group Design, illustrated above. The design includes at least an experimental and control group. It mirrors the Pretest-Posttest control group experimental design, but instead of randomization, naturally occurring comparison groups are selected to be as alike as possible (Gribbons & Herman, 1997).
With a Time Series Design, observations are taken (in this case three) to establish a baseline; a treatment then occurs, X, followed by additional observations being made. From this, an estimate of the impact the treatment made is computed (Gribbons & Herman, 1997; Morgan, Gliner & Harmon, 2000). The design can be employed to establish a baseline measure, describe a change over time or to keep track of trends. Data are almost always presented in a graph.
The Multiple Time Series Design is simply an extension of the Time Series Design with the addition of a comparison group. It attempts to model what would have happened to the experimental group if the treatment or intervention had not taken place. The addition of the control adds credibility, even without randomization.
In the Equivalent Materials Design, different, equivalent materials are represented throughout with Ma,b,c,d. The treatments and repeats of treatments, X0,1, are applied and then observed. In addition to the aforementioned designs, there are elaborate extensions, such as the Latin-square Design (Fortune & Hutson, 1984).
It was not until the late 1900s that educational reform gave credence to research. Prior to that, untested interventions and innovations were commonplace. The U.S. government has come to demand a research base, as demonstrated by the Comprehensive School Reform Demonstration legislation of 1997 and the No Child Left Behind Act. In both cases, the emphasis is in the application of experimental or quasiexperimental research (Slavin, 2003).
Campbell and Stanley (1963) speak to traditional experimental and quasiexperimental methods. Cook and Campbell (1979) continued to explore these methods, building upon early investigations of validity. In 1993 Parker reviewed and synthesized both the works and further added to the mound of threats to validity. At the turn of the century, various authors were continuing to write about the issue of what constitutes an experimental design and the issues associated with a less than perfect experiment. The presence of true experimental designs in educational research has not been prevalent over time largely due to methodological constraints and on some accounts, more logistical issues like money and time. It is frequently not practical to randomly assign individuals to control and experimental groups, like in a clinical setting where groups are already intact (Dimsdale & Kutner, 2004; Heppner, Kivlighan & Wampold, 1992). Quasiexperimental designs offer a plausible solution to these dilemmas.
Other issues that typically surface in educational research include, but are not limited to, confounding and the assumption of independence. Independence states that the measure for an individual is independent of the measures of other individuals. Randomization of subjects is the only way to insure this. Confounding is another concern. When a variable in a research design is not controlled, but should be, it is identified as a confounding factor. Given that confounding cannot be dealt with in terms of statistical notions alone, quasiexperi-mental designs, and for the most part educational research as a whole, always have this as a limitation (Pearl, 2000).
If evidence-based school reform continues to be the golden standard, quasiexperimental research will play an instrumental role. Quasiexperimental research, and the scientifically based results that go along with it, can provide the educational community with a variety of models that have been shown to be effective. Reading First and other similar initiatives have created evidence-based reform that can be sustained. Because of this, it is plausible that the rigor associated with quasiexperimental designs will become commonplace (Slavin, 2003; U.S. Department of Education, 1998; U.S. Department of Education, 2002).
Valentin (1997) made a claim that with an understanding of eight statistical procedures, it is reasonable to have an understanding of 90% of quantitative research. Experimental designs lend themselves to straightforward, often simpler, statistical analysis than quasiexperimenatal designs. Advanced statistical procedures are typically necessary in quasiexperimental research, largely due to the lack of randomization (Dimsdale & Kutner, 2004).
Two specific examples include multiple regression analysis and factor analysis. Multiple regression analysis is a statistical application that is utilized in studies in which impact is being measured. Using statistical methods, a control group is simulated, and multiple adjustments can be made for outside factors. Thus, the control that is in the design of an experiment is inserted through analytical techniques (SERVE Center, 2007). Factor analysis is a useful technique when a study has a large number of variables. This statistical application allows for a reduction in the number of variables while detecting possible relationships between those variables of interest (Dimsdale & Kutner, 2004). It is commonly applied when data is collected through a survey, especially when the survey contains a large number of items. Analysis of covariance (ANCOVA) is yet another analytical technique employed to increase the strength of the quasiexperimental design. By making compensating adjustments, ANCOVA reduces the effects of the initial differences between groups. This again is an attempt to respond to the lack of randomization.
When considering what type of design to employ in a study, it is important to consider both validity and practicality. In general, quasiexperimental research is more feasible, given the typical time and logistical constraints. At the surface level, an easily identifiable weakness of employing quasiexperimental research, in contrast to a true experiment, is the lack of random assignment. Without random assignment, internal validity is reduced, and causal claims become quite difficult to make (Prater, 1983).
On the other side, quasiexperimental designs tend to present the situation under investigation in real-world conditions, increasing the external validity. Typically, quasiexperimental designs are pre-existing constructions. Because of this, fewer variables are able to be controlled; yet another factor limiting the ability to make causal claims (Henrichsen, Smith & Baker, 1997).
With the implementation of the No Child Left Behind statute, educational research put forth an agenda of scientifically based research. Shavelson and Towne (2002) outline criteria necessary for a scientific study, which include: direct, empirical investigation of an important question; consideration for the context in which the study took place; alignment with a conceptual framework; careful and thorough reasoning; and disclosure of results. Quasiexperimental research makes the mark by meeting each criterion listed. While the controlled, experimental design is the ideal, at least statistically, when an experiment is not possible or practical, the best approach is to identify and eliminate threats to validity through the implementation of a quasiexperi-mental approach (Borg & Gall, 1989).
See also:Research Methods: An Overview
Borg, W. R., & Gall, M. D. (1989). Educational research: An introduction. New York: Longman.
Campbell, D., & J. Stanley. (1963). Experimental and quasi-experimental Designs. Chicago: Rand McNally.
Cook, T., & D. Campbell. (1979). Quasi-experimental design. Chicago: Rand McNally.
Dawson, T. E. (1997, January 23–25). A primer on experimental and quasi-experimental design. Paper presented at the Annual Meeting of the Southwest Educational Research Association, Austin, TX.
Dimsdale, T., & Kutner, M. (2004, December 9–11). Becoming an educated consumer of research: A quick look at the basics of research methodologies and design. Meeting of the Minds Practitioner-Researcher Symposium. American Institutes for Research, Sacramento, CA.
Fortune, J. C., & Hutson, B. A. (1984). Selecting models for measuring change when true experimental conditions do not exist. Journal of Educational Research, 77(4), 197–206.
Gribbons, B., & Herman, J. (1997) True and quasi-experimental designs. Washington, DC: ERIC Clearinghouse on Assessment and Evaluation.
Henrichsen, L., Smith, M. T. & Baker, D. S. (1997). Taming the Research Beast, Brigham Young University. Retrieved April 16, 2008, from http://linguistics.byu.edu/faculty/henrichsenl/researchmethods/RM_0_02.html.
Kidder, L. H. (1981). Research methods in social relations. Fort Worth, TX: Holt, Rinehart & Winston.
Morgan, G. A., Gliner, J. A., & Harmon, R. J. (2000). Quasi-experimental designs. Journal of the American Academy of Child & Adolescent Psychiatry, 39(6), 794–796.
Pearl, J. (2000). Causality: Models, reasoning, and inference. Cambridge, England: Cambridge University Press.
Prater, J. M. (1983). An analysis of selected statistical techniques utilized in quasi-experimental designs. Paper presented at the Annual Meeting of the Mid-South Educational Research Association, November 16–18, Nashville, TN.
Schoenfeld, A. H. (2006). Design Experiments. In Judith L. Green, Gregory Camilli, & Patricia B. Elmer (Eds.), Handbook of complementary methods in educational research (pp. 193–205). Mahwah, NJ: Erlbaum.
SERVE Center. (2007). Education Research: What is scientifically based research? University of North Carolina, Greensboro. Retrieved April 16, 2008, from http://www.serve.org/EdResearch/SBR/quasi.php.
Shavelson, R. J., & Towne, L. (2002). Scientific research in education. Washington, DC: National Academy Express.
Slavin, R. E. (February, 2003). A reader's guide to scientifically based research. Educational Leadership, 12–16.
U.S. Department of Education. (1998). Guidance on the comprehensive school reform demonstration program. Washington, DC: Author.
U.S. Department of Education. (2002, June 14). Draft guidance on the comprehensive school reform program. Washington, DC: Author.
Valentin, T. (1997). Understanding quantitative research about adult literacy. Focus on basics, 1(A), Cambridge, MA: National Center for the Study of Adult Literacy and Learning. Retrieved April 16, 2008, from http://www.ncsall.net/?id=470.
Washington Virtual Academies
Tuition-free online school for Washington students.