Traditional scores versus IRT estimates on forced-choice tests based on a dominance model

Résumé 0

"Pedro M. Hontangas, Iwin Leenen, Jimmy de la Torre, Vicente Ponsoda, Daniel Morillo and Francisco J. Abad 76 Forced-choice tests (FCTs) have a long tradition in psychology, and were proposed to address response bias in Likert-type items (Christiansen, Burns, & Montgomery, 2005; McCloy, Heggestad, & Reeve, 2005). FCTs are generally composed by blocks of items with different items measuring different dimensions (Multidimensional, MFCTs) or the same dimension (Unidimensional, UFCTs). The most common formats are called PICK, MOLE and RANK (Hontangas et al., 2015). The PICK format instructs respondents to choose the item in the block that is most descriptive of them; in the MOLE format, they choose the most as well as the least descriptive item; and when the RANK format is used, respondents rank all the items from most to least descriptive (for examples, see Hontangas et al., 2015; and for a condensed literature review on FCTs, see van Eijnatten, van der Ark, & Holloway, 2015). Empirical evidence shows that MFCTs, to some extent, can control response biases (e.g., Cheung & Chan, 2002; Saville & Willson, 1991), increase criterion validity (Bartram, 2007; Brown & Maydeu-Olivares, 2013), and avoid or reduce faking (Christiansen et al., 2005; Heggestad, Morrison, Reeve, & McCloy, 2006; Hirsh & Peterson, 2008; Jackson, Wroblewski, & Ashton, 2000). However, MFCTs have been criticized because traditional scoring methods yield ipsative or partially ipsative scores and typically result in incorrect estimates of reliability, scale intercorrelations, and factor loadings, and in unwarranted (normative) interpretations ISSN 0214 - 9915 CODEN PSOTEG Copyright © 2016 Psicothema www.psicothema.com Traditional scores versus IRT estimates on forced-choice tests based on a dominance model Pedro M. Hontangas 1 , Iwin Leenen 2 , Jimmy de la Torre 3 , Vicente Ponsoda 4 , Daniel Morillo 4 and Francisco J. Abad 4 1 Universidad de Valencia, 2 Universidad Nacional Autónoma de México, 3 The State University of New Jersey (USA), and 4 Universidad Autónoma de Madrid Abstract Resumen Background: Forced-choice tests (FCTs) were proposed to minimize response biases associated with Likert format items. It remains unclear whether scores based on traditional methods for scoring FCTs are appropriate for between-subjects comparisons. Recently, Hontangas et al. (2015) explored the extent to which traditional scoring of FCTs relates to the true scores and IRT estimates. The authors found certain conditions under which traditional scores (TS) can be used with FCTs when the underlying IRT model was an unfolding model. In this study, we examine to what extent the results are preserved when the underlying process becomes a dominance model. Method: The independent variables analyzed in a simulation study are: forced-choice format, number of blocks, discrimination of items, polarity of items, variability of intra-block dif fi culty, range of dif fi"