Assessment Design for International Large-Scale Assessments

Authored by: Leslie Rutkowski , Eugene Gonzalez , Matthias von Davier , Yan Zhou

Handbook of International Large-Scale Assessment

Print publication date:  November  2013
Online publication date:  November  2013

Print ISBN: 9781439895122
eBook ISBN: 9781439895146
Adobe ISBN:

10.1201/b16061-7

 Download Chapter

 

Abstract

As an invariant constraint to producing high-quality measurement instruments, questionnaire length is a significant factor in the development process. Owing to issues of fatigue, attrition, or the logistics of available study participants, instrument (referred to hereafter as test) developers must constantly weigh measuring the construct(s) of interest in a valid and reliable way against the real limitations associated with administration. To that end, a method referred to as multiple matrix sampling (MMS) is used in large-scale educational surveys such as the Programme for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS) to balance the demands of test length against the need for sound measurement. Although methods for estimating latent traits from multiple matrix designs are the subject of much current or recent methodological research (e.g., Thomas and Gan 1997; von Davier and Sinharay 2007, 2009), advances in MMS theory, outside of mathematically optimizing designs (e.g., van der Linden and Carlson 1999; van der Linden et al. 2004) have largely come to a slowdown. Further, much of the primary literature on MMS has lain dormant for many years or, excepting a few recent papers (e.g., Childs and Jaciw 2003; Frey et al. 2009; Gonzalez and Rutkowski 2010), ascetic discussions around the matter have been relegated to the pages of technical manuals. Therefore, we pick up this discussion by detailing the origins of MMS, describing where it has been applied in educational assessments and who currently uses it (and why), and we suggest how this particular assessment design might be applied in new ways in the future. We focus many of our views for the future on international, national, and state-level large-scale assessments; however, the potential of MMS is not limited to these contexts.

 Cite
Search for more...
Back to top

Use of cookies on this website

We are using cookies to provide statistics that help us give you the best experience of our site. You can find out more in our Privacy Policy. By continuing to use the site you are agreeing to our use of cookies.