Test Problems: Seven Reasons Why Standardized Tests Are Not Working

Inner-city schools aren't the only ones experiencing testing woes; rural communities and wealthy suburbs have their own complaints. In Scarsdale. New York, an upscale, college-oriented community, parents organized a boycott of the eighth-grade standardized tests. Of 290 eighth-graders, only 95 showed up for the exam. In Miami protests erupted when over 12,000 Florida seniors were denied their high school diploma, and in Massachusetts, local school boards defied the state and issued their own diplomas to students they believed were being unfairly denied their high school graduation because of the state-mandated test. Teachers in California and Chicago refused to give tests and faced disciplinary action. Why are teachers, students, and parents protesting? What's wrong with measuring academic progress through standardized tests? Here are some reasons why high-stake tests are problematic:

1. At-Risk Students Placed at Greater Risk. Using the same tests for all students, those in well-funded posh schools along with students trying to learn in under-funded, ill-equipped schools is grossly unfair, and the outcome is quite predictable. Since students do not receive equal educations, holding identical expectations for all students places the poorer ones at a disadvantage. State data confirm that African Americans and Hispanics, females, poor students and those with disabilities are disproportionately failing "high-stakes" standardized tests. In Louisiana, parents requested that the Office for Civil Rights investigate why nearly half the students in school districts with the greatest numbers of poor and minority children had failed Louisiana's test, even after taking it for a second time. In Georgia, two out of every three low-income students failed the math, English, and reading sections of the state's competency tests. No students from well-to-do counties failed any of the tests and more than half exceeded standards. Even moderate income differences could result in major test score differences. In Ohio, almost half of the students from families with incomes below $20,000 failed the state exams, while almost 80 percent of students from families earning more than $30,000 passed those same exams.

A third kind of standard called an opportunity-to-learn standard, was supposed to remedy these social and economic challenges. Opportunity-to-learn standards were to ensure a level playing field by providing all students with appropriate educational resources, competent teachers and modern technology. Differences in student learning styles were to he accommodated and additional time provided for students to relearn material if they failed the test. Teachers were to be given quality in-service training. Yet half the states did not earmark money to remedy dramatic educational differences between school districts, and real barriers to achievement—racism, poverty, sexism, low teacher salaries, language differences, inadequate facilities—were lost in the sea of testing. Although the rhetoric of the standards movement is that a rising tide raises all ships, in fact, without the adequate resources, some ships do not rise. In the current standards and testing movement, opportunity-to-learn is the forgotten standard.

2. Lower Graduation Rates. Grade-by-grade testing and graduation tests actually increase school dropouts. A Harvard University study found that students in the bottom 10 percent of achievement were 33 percent more likely to drop out of school in states with graduation tests. The National Research Council found that low-performing elementary and secondary school students who are held back do less well academically, are much worse off socially, and are far likelier to drop out than equally weak students who are promoted. Retention in grade is the single strongest predictor of which students will drop out—stronger even than parental income or mother's education level.

Education Secretary Paige was given credit for dramatic improvement of test scores in Houston, where he was superintendent. Houston was the center-piece of the "Texas Miracle" and the foundation for No Child Left Behind. But by 2003, it became clear that underreported dropouts contributed to test score gains. Houston reported a dropout rate of just over 1 percent a year, but that statistic was put in doubt by later studies that found the dropout rate was closer to 40 percent. When poor students, Latinos, African Americans, and Native Americans fail to meet graduation testing requirements, they are retained in grade and then likely to drop out. When their low scores disappear, the school's average test score improves, giving a picture of success when the real picture is failure

3. Higher Test Scores Do Not Mean More Learning. Evidence is mounting that for a growing number of schools, teaching is being redefined as test preparation. Seventy-nine percent of teachers surveyed by Education Week said they spent "a great deal" or "somewhat" of their time instructing students in test-taking skills, and 53 percent said they used state practice tests a great deal or somewhat. In Texas, James V. Hoffman and his colleagues asked reading teachers and supervisors to rate how often they engaged in test preparation. The study used a scale of 1 to 4, in which 1 stood for never, 2 for sometimes, 3 for often, and 4 for always. Most of those surveyed said that teachers en-gaged in the following activities "often" or "always":

• Teaching test-taking skills—3.5
• Having students practice with tests from prior years—3.4
• Using commercial test preparation materials—3.4
• Giving general tips on how to take tests—3.4
• Demonstrating how to mark an answer sheet correctly—3.2

In one school, for example, students were taught to cheer "Three in a row? No, No, No!" The cheer was a reminder that if students answered "c" three times in a row, probably at least one of those answers is wrong since the test maker is unlikely to construct three questions in a row with the same answer letter.

Although this kind of test preparation may boost scores, it does not necessarily produce real gains in understanding that show up on other tests or performance measures or that students can apply in a nontesting situation. Consider these findings:

• A study of eighteen states with high-stakes testing compared trends in state test scores with long-term trends on other standardized tests. When state tests were given, other test scores often dropped. In more than half of these states, performance went down on the ACT, SAT, and the math test of the National Assessment of Educational Progress (NAEP). The study concluded that higher state test scores were most likely due to direct test preparation rather than increased student learning, and to differences in how many students were excluded from testing based on disabilities or limited English proficiency.
• Three-quarters of fourth-grade teachers surveyed by RAND in Washington State, and the majority of principals, believed that better test preparation (rather than increased student knowledge) was responsible for most of the score gains.
• In Kentucky's state assessment, scores went up on test items that were reused, then dropped when new items were introduced. This discrepancy between new and reused items was larger in schools that had greater over-all test score gains, a relationship that suggests students were being coached on reused items.

4. Standardized Testing Shrinks the Curriculum. Educator Alfie Kohn advises parents to ask an unusual question when a school's test scores increase: "What did you have to sacrifice about my child's education to raise those scores?" As schools struggle to avoid the "underperforming" label, entire subject areas—such as music, art, social studies, and foreign languages—are de-emphasized. What is not tested does not count, and 85 percent of teachers believe that their school gives less attention to subjects that are not on the state test. One teacher had this to say about how the timing of state tests drives teaching: "At our school, third- and fourth-grade teachers are told not to teach social studies and science until March." As "real learning" takes a backseat to "test learning," challenging curriculum is replaced by multiple choice materials, individualized student learning projects disappear, and in-depth exploration of subjects along with extracurricular activities are squeezed out of the curriculum.

5. When Tests Fail. Tests themselves are often flawed, and high-stakes errors be-come high-stakes disasters. When Martin Swaden's daughter failed the state math test by a single answer, Swaden requested to see the exam so that he could help his daughter correct her errors and pass the test next time around. It took a threatened lawsuit before he was able to meet with a state official to ex-amine the answers. Together they made an amazing discovery: six of the sixty-eight answers were keyed incorrectly, not only for his daughter, but for all the students in Minnesota. Jobs had been lost, summers ruined, the joy of graduation turned to humiliation for those students who were misidentified as having failed. Suits followed and $7 million in damages were eventually paid, but the testing company argued that it was not liable for "emotional damages."

Unfortunately, such stories continue to mount as the crush of millions of new tests overwhelms the handful of testing companies. In Massachusetts, a senior spotted an alternative answer to a math question, and the scores of 449 students were suddenly propelled over the passing mark. A Massachusetts teacher spotted a question with two correct answers, and when the scores were adjusted, 666 more students passed. A flawed answer key incorrectly lowered multiple-choice scores for 12,000 Arizona students, erred in adding up scores of essay tests for students in Michigan and forced the re-scoring of 204,000 essay tests in Washington. Another error resulted in nearly 9,000 students in New York City being mistakenly assigned to summer school, and $2 million in achievement awards being denied to deserving students in Kentucky. By 2003, the National Board on Educational Testing and Public Policy reported that fifty high-profile testing mistakes had occurred in twenty states from 1999 through 2002.

Test materials also have been delivered late, or with missing pages, or errors have been found in scoring. Raters complained that they are given inadequate training and little time to grade essays, and at $9 an hour, many doubt the accuracy of their ratings. Many question the wisdom of rewarding and punishing students, teachers, and schools based on the flawed history of the testing industry.

6. Teacher Stress. While teachers support high standards, they object to learning being measured by a single test. Not surprisingly, in a national study, nearly seven in ten teachers reported feeling test-stress, and two out of three believed that preparing for the test took time from teaching important but non-tested topics.' Fourth-grade veteran teachers were requesting transfers, saying that they could not stand the pressure of administering the high-stakes elementary exams, and teachers recognized for excellence were leaving public schools, feeling their talents were better utilized in private schools where test preparation did not rule the curriculum. When eighty Arizona teachers and teacher educators were asked to visually depict the impact of standardized tests, their drawings indicated test-driven classrooms where boredom, fear, and isolation dominate. Teachers feel that they are shortchanging schoolchildren from a love for learning. Figure 10.4 presents one of those drawings. (For others, visit Mr. Tirupalavanam Ganesh's Web site at ganesh.ed.asu.edu/aims.)

7. What's Worth Knowing? The fact that history, drama, the arts and a host of subjects are given less attention in the current testing movement raises intriguing curricular questions: What is really important to teach'? What is worth knowing? While it may sound pretty obvious, thinking beyond the obvious is often a good idea. Much of what is taught in schools is tradition and conventional wisdom, curricular inertia rather than careful thought. To see how society's notion of what is important can change, try your hand at the following test questions that were used to make certain that eighth graders in Kansas knew "important information." We have shortened the exam, but all these questions are from the original. (Flint: Brush up on your orthography.) See if you would qualify to graduate from elementary school in 1895.