Criterion and Standards-Referenced Tests

Standards-Referenced Tests

A recent variation of criterion-referenced testing is "standards-referenced testing" or "standards based assessment." Many states and districts have adopted content standards (or "curriculum frameworks") which describe what students should know and be able to do in different subjects at various grade levels. They also have performance standards that define how much of the content standards students should know to reach the "basic" or "proficient" or "advanced" level in the subject area. Tests are then based on the standards and the results are reported in terms of these "levels," which, of course, represent human judgement. In some states, performance standards have been steadily increased, so that students continually have to know more to meet the same level.

Educators often disagree about the quality of a given set of standards. Standards are supposed to cover the important knowledge and skills students should learn -- they define the "big picture." State standards should be well-written and reasonable. Some state standards have been criticized for including too much, for being too vague, for being ridiculously difficult, for undermining higher quality local curriculum and instruction, and for taking sides in educational and political controversies. If the standards are flawed or limited, tests based on them also will be. In any event, standards enforced by state tests will have -- and are meant to have -- a strong impact on local curriculum and instruction.

Even if standards are of high quality, it is important to know how well a particular test actually matches the standards. In particular, are all the important parts of the standards measured by the test? Often, many important topics or skills are not assessed.

A major reason for this is that most state exams still rely almost entirely on multiple-choice and short-answer questions. Such tests cannot measure many important kinds of learning, such as the ability to conduct and report on a science experiment, to analyze and interpret information to present a reasonable explanation of the causes of the Civil War, to do an art project or a research paper, or to engage in serious discussion or make a public presentation (see fact sheet on multiple-choice tests). A few standards-based exams have gone beyond multiple-choice and short-answer, but even then they may not be balanced or complete measures of the standards.

CRTs and NRTs

Sometimes one kind of test is used for two purposes at the same time. In addition to ranking test takers in relation to a national sample of students, a NRT might be used to decide if students have learned the content they were taught. A CRT might be used to assess mastery and to rank students or schools based on their scores. In many states, students have to pass either an NRT or a CRT to obtain a diploma or be promoted. This is a serious misuse of tests. Because schools serving wealthier students usually score higher than other schools, ranking often just compares schools based on community wealth. This practice offers no real help for schools to improve.

NRTs are designed to sort and rank students "on the curve," not to see if they met a standard or criterion. Therefore, NRTs should not be used to assess whether students have met standards. However, in some states or districts a NRT is used to measure student learning in relation to standards. Specific cut-off scores on the NRT are then chosen (usually by a committee) to separate levels of achievement on the standards. In some cases, a CRT is made using technical procedures developed for NRTs, causing the CRT to sort students in ways that are inappropriate for standards-based decisions.

Sometimes the NRT is changed to more closely fit the state standards and to report standards- referenced scores. As a result, a state could report that 35 percent of its students were proficient according to state standards (depending, of course, on where the cut-off score is set), but that 60 percent of its students were above the national average score on the norm-referenced test. Adapting an NRT also means that while everything on the test is in the standards, much of what is in the standards is not in the tests.

Conclusion

If standardized tests are used at all, CRTs make more sense for schools than do NRTs. However, they should be based on relevant, high-quality standards and curriculum and should make the least possible use of multiple-choice and short-answer questions. As with all tests, CRTs and NRTs, no matter what they are called, should not control curriculum and instruction, and important decisions about students, teachers or schools should not be based solely or automatically on test scores.