Due to the high-stakes nature of large-scale assessment, it is sensible to ensure that results from the assessments are based on effective instruction and true student achievement. U.S. Secretary of Education Arne Duncan issued a policy letter, dated June 24, 2011, that urges states to “make assessment security a high priority” and to “ensure that assessment development contracts include support for activities related to test security, including forensic analysis.” Additionally, it is recommended by the Association of Test Publishers and the Council of Chief State School Officers (2010) that rules and procedures be adopted that respond to instances of test administration irregularities.
To assist in meeting these recommendations, DRC protects its clients’ investments in testing through the use of its industry-leading research on erasure behavior, expert knowledge of the tests’ underlying measurement models, and other statistical indicators of testing irregularities.
DRC’s expertise means clients have access to the industry’s best tools to continuously protect the integrity of their assessment and accountability programs.
Scanning technology used by DRC provides detailed erasure analysis at the student level. This analysis takes data from the multiple-choice item responses of all students and analyzes it at the state, school, and student levels for evidence of responses being erased and changed.
Erasures are not made when students are taking a computer-based assessment. Answer changes, however, are still made. DRC’s online test engine, DRC INSIGHT, is able to track student progress throughout the test administration. If a student changes a multiple-choice answer, the system is able to track the answer that was initially given, what answer was finally selected, and everything in between.
Statistical indices are used to identify potential testing irregularities and separate meaningful gains from the spurious.
Use of Multiple Measures
DRC uses a variety of traditional and advanced statistical analyses to detect aberrances in test-taking behavior. These techniques provide for improved prediction and increased sensitivity. The methods used are as follows:
Answer Change Analysis
Score Fluctuation Analysis
Rasch Residual Analysis
Response Time Analysis
Student Participation Analysis