One of the great gifts of standardized testing was the premise of giving the same questions to many students would allow for a more reliable measure of student achievement. This reliability is gained through standardization of the test, while validity is gained by the relevance of the questions to the stated purpose of the test. In the 1920s, college admission officers leaped on this bandwagon in order to compare students from New Hampshire and Ohio, and thus was born our accountability system tied to test scores.
However, there are some problems with this logic. What if the students had not been prepared in the same way? What if there were cultural reasons why answers might differ across state lines? What if smart teachers or test-coaching companies could study the test and provide useful insight? And what if test companies manipulated the pass/fail line, commonly called the cut score, for political reasons? In a recent article in Education Next, Michael J. Petrilli discusses the illusion of proficiency and the resulting gap in honesty: