Factors That Influence Reliability

Several factors can affect the reliability of a test (Mehrens & Lehmann, 1991; Sattler, 2001):

1. Test length. Generally, the longer a test is, the more reliable it is.
2. Speed. When a test is a speed test, reliability can be problematic. It is inappropriate to estimate reliability using internal consistency, test-retest, or alternate form methods. This is because not every student is able to complete all of the items in a speed test. In contrast, a power test is a test in which every student is able to complete all the items.
3. Group homogeneity. In general, the more heterogeneous the group of students who take the test, the more reliable the measure will be.
4. Item difficulty. When there is little variability among test scores, the reliability will be low. Thus, reliability will be low if a test is so easy that every student gets most or all of the items correct or so difficult that every student gets most or all of the items wrong.
5. Objectivity. Objectively scored tests, rather than subjectively scored tests, show a higher reliability.
6. Test-retest interval. The shorter the time interval between two administrations of a test, the less likely that changes will occur and the higher the reliability will be.
7. Variation with the testing situation. Errors in the testing situation (e.g., students misunderstanding or misreading test directions, noise level, distractions, and sickness) can cause test scores to vary.