The current invention is designed to improve the certainty of test-taker
classifications in testing programs utilizing open-ended items. The
current invention discloses methods to improve certainty in estimates of
test-taker performance for all test-takers while using a minimum of
scoring resources. The invention selectively allocates resources to
scoring by ceasing to allocate scoring resources to test-takers whose
performance has been determined with sufficient certainty, and by having
an additional reader or readers score responses from test-takers whose
scores are close to cutoff levels. The invention further increases
precision of scores for test-takers near a cutoff by selectively
allocating more reliable and accurate readers to score responses most
likely to influence a test-taker's performance classification. Other
scoring resources are also shown.