A Position Sensitive IRT Mixture Model [post]

Klint Kanopka, Benjamin Domingue
2022 unpublished
Standard item response theory (IRT) models are ill-equipped for scenarios wherein theprobability of a correct response exhibits a dependence on the location in the test where anitem is encountered—a phenomenon that we broadly refer to as position effects.Unmodeled position effects make observed performance contingent upon the specific testform a respondent is exposed to, introducing a potential liability in our attempt to measureability, directly compare students taking the same test, and make
more » ... onsequential inferencesfrom test scores. Traditional work on position effects tends to model them as a purelyitem-side or person-side phenomenon, but observations of within-person variation inresponse time within a single computer adaptive test implies respondents may be engagedin an evolving response process over the course of a test (Domingue et al., 2021). Inspiredby the notion of an evolving response process, we propose a novel position sensitive IRTmodel that is a mixture of item response models. The entire model is a mixture of twoitem response functions that separately capture the difference in response probability whenthe item is encountered early versus late in the test. The mixing proportions of the twomodels depend both on the location of the item in the test booklet and estimatedperson-level characteristics. By imposing a specific functional form on the mixingparameter, we separate person and item contributions to position sensitivity. We describesimulation studies outlining various features of model performance and end with anapplication to a large scale admissions test with observed position effects.
doi:10.31234/osf.io/hn2p5 fatcat:zx3zvcklojgy3pu7b34entz7ca