Frequently Asked Questions about the WASL (continued)

9. How are passing scores determined?

This process is called “standard setting” because students must meet a certain performance standard to pass. Setting standards for the WASL is a thoughtful and involved process incorporating the feedback of many people. First, a standard-setting panel for a WASL subject and grade level is convened. Members include teachers, parents and community members representing Washington’s geographically and ethnically diverse population.

The panelists for each content area review "performance-level descriptors" – the written descriptions of what students should know and be able to do in that subject and grade. They also look at the test itself to see how many points a student should earn on the test to meet the “performance-level descriptors.” The panel’s work is done in rounds. After the first round of deliberations, panelists discuss each others’ perspectives and then conduct a second round of review. A third round is done before the panel, as a whole, makes final recommendations.

Next, an “articulation” committee is brought together to ensure the suggested standards relate sensibly to one another across the different grade levels. The articulation committee members represent standard-setting committee members from different subjects and grades. The articulation committee reviews the standard-setting committee’s recommendations and can make its own set of recommendations.

Both the recommendations from the standard-setting panel and articulation committee are forwarded to the State Board of Education for review and adoption. Once the State Board of Education decides which recommendation to adopt, that is the performance a student must achieve in order to “meet standard” or pass the WASL.

10. Are standards reset each year?

No. Once the State Board of Education adopts a set of standards for a WASL test, the state carries that expected level of performance from year to year. Each year a new edition of the test is developed. Most of the questions on the test are new questions, but some of the questions have appeared in earlier years. The repeated items are “anchor” items that are used to link the performance on one year’s edition of the WASL to earlier editions of the test. This uses a procedure called “equating.” By equating the current year’s WASL to WASLs given in previous years, the performance standard can be maintained over time.

11. What is a good score on the WASL?

A student’s performance on the reading, math and science WASLs is reported using “scale scores.” Scale scores are three-digit numbers that are used to place the student into one of four levels: Advanced (Level 4), Proficient (Level 3), Basic (Level 2) and Below Basic (Level 1). A scale score of 400 is assigned to a student who has just barely met the state standard; this score is at the lower end of Level 3. Students scoring in Level 4 are said to have exceeded the state standard. Students with scores in Level 1 or Level 2 have not met standard.

Students generally have to achieve a score that represents approximately 60 to 65 percent of the points possible on each test to pass. That score or above means they have met the required standard for proficiency in that particular subject.

View each level’s score range by grade and content area.

12. What steps are taken to make sure that the scoring of open-ended items is valid and reliable?

The Office of Superintendent of Public Instruction (OSPI) uses rigorous methods to ensure that the scoring process yields valid and reliable results. Valid scoring means that a scorer assigns the same score to a student response as would be assigned by an expert panel of Washington educators. Reliable scoring means that different scorers consistently assign the same score to a student response.

The following quality control measures are used when open-ended questions are scored on the WASL:

• Item-by-item scoring: Teams of scorers are trained to score responses to a single open-ended WASL question and all responses to that question are scored by the team. Once all students’ responses to that question have been scored, the team is trained to score a different test question. This process adds to the scoring consistency, as scorers need only keep a single scoring guide in mind as they score. This approach also protects against scorer biases that could come into play if entire test booklets were scored by a single scorer. For math, reading and science assessments, scorers may be trained on 2 or 3 items and “rotate” through those 2 or 3 items either daily or every other day. This is a safeguard against scorer fatigue.
• Double-scoring: At least 10 percent of all writing responses in grades 4 and 7 and 100 percent of high school writing papers are scored twice to verify that scoring is consistent and aligned to the scoring rubrics. A minimum of one out of 20 of all responses to open-ended questions in math, reading and science are double-scored.
• Supervisors Reread Scored Student Work: In addition to double-scoring, each scoring supervisor rereads an average of five percent of the papers from scorers under their supervision every day. If a supervisor discovers that a scorer begins to assign scores that do not match the scoring guidelines, the supervisor consults with the scoring director and together they retrain that scorer, using the original training materials. This on-the-spot checking helps keep the scoring consistent. If a scorer has drifted from the scoring guidelines, the scores he/she has recently-assigned are removed and those papers are reinserted into the queue to be rescored. Scorers who prove unable to score consistently after retraining are dismissed from scoring.
• Blindly Inserted Validity Papers: A "validity paper" is a student response that has been pre-approved by Washington content (reading, writing, math and science) specialists as being a clear example of a score point. Multiple validity papers are blindly inserted into a scorer's assignment of student responses to be scored. Scoring Supervisors receive a daily report of how well scorers' decisions matched with the pre-determined score on the validity papers. Any variation from the scoring criteria is addressed immediately.
• Protocols to Handle Unique Responses: Scorers are trained to only assign a score to student responses that are consistent with the examples provided in training. If a scorer encounters a student response that is unique, novel or otherwise unfamiliar, the scorer seeks advice from the Supervisor. If the response is new to the Supervisor, the Scoring Director intervenes. At this point, the Scoring Director can decide either that the response is merely a nuance of what is already described in the scoring rubric or that the response is truly unique. If the response is a nuance, all the scorers are notified and re-trained on that particular type of response; if the response is one that has not yet been encountered, OSPI content staff intervene and determine what score should be assigned, after which scorers are re-trained.
• Communication between OSPI and the Contractor: OSPI representatives are on site at Pearson’s facilities during training and much of scoring to monitor the quality processes and address any content questions that may surface.

13. How are test results reported?

Results are reported for individual students, schools, districts and the state according to four performance levels defined by the State Board of Education:
Level 4 – Advanced
Level 3 – Proficient
Level 2 – Basic
Level 1 – Below Basic