Identifying and Investigating Aberrant Responses using Psychometrics-Based and Machine Learning-Based Approaches

Authored by: Doyoung Kim , Ada Woo , Phil Dickison

Handbook of Quantitative Methods for Detecting Cheating on Tests

Print publication date:  October  2016
Online publication date:  October  2016

Print ISBN: 9781138821804
eBook ISBN: 9781315743097
Adobe ISBN: 9781317588108

10.4324/9781315743097.ch4

 Download Chapter

 

Abstract

A test is high-stakes if its results are used to make important decisions about test takers. In general, test takers take high-stakes tests to receive benefits such as a license to practice an occupation or to avoid punishments (e.g., being denied a diploma, not being permitted to drive a car). As more high-stakes tests are administered to more people for diverse reasons, the validity of test scores becomes important. It is imperative that any person or organization who uses high-stakes test scores to make such important decisions about test takers be confident that the scores from the test takers are reliable and provide valid indications of the test takers’ abilities on the construct being measured by the test. In the recent International Testing Committee (ITC) guidelines on quality control in scoring, test analysis, and reporting of test scores (International Test Commission, 2013), it is recommended that aberrant or unexpected response patterns (e.g., missing easy items while answering difficult ones correctly) should be monitored routinely through statistical techniques for detecting invalid test scores. Because the aberrant item response patterns have a negative impact on the validity of test scores, it is important to identify and investigate these item response patterns prior to drawing any conclusion using the test scores. As computer-based testing has become popular, test takers’ response times are readily available, and having item responses accessible can facilitate detection of invalid test scores. These response time data are not only important themselves in that aberrant response time patterns can be identified by fitting statistical models to the response time data but are also valuable to bring a new perspective to investigating aberrance when using together with the response pattern data. Furthermore, some of the auxiliary information collected with item response and response time data, such as test takers’ demographic variables and testing center environment, could also inform any investigation into the individual examinees whose responses are aberrant.

 Cite
Search for more...
Back to top

Use of cookies on this website

We are using cookies to provide statistics that help us give you the best experience of our site. You can find out more in our Privacy Policy. By continuing to use the site you are agreeing to our use of cookies.