Mutual Information as a Tool to Identify Rapid-Responding Behaviour – Santeri Holopainen (Turku Research Institute for Learning Analytics)

In large-scale assessments, test-taker disengagement causes a response process in which a test-taker does not show their true knowledge, skill, or ability, hindering the validity of the observed test score. Due to the emergence of computer-based tests (CBTs), which allow the test administrator to measure test-takers’ response times (RTs), disengagement research has focused on identifying rapid-responding behaviour (RRB), a response process in which a test-taker answers an item too quickly to have fully considered it. In contrast, if the test-taker considered the item fully, they would exhibit solution behaviour (SB). In applied research and operational settings, using response time threshold methods (RTTMs) to find item-specific time thresholds that best differentiate between RRB and SB has been the most common approach to identify RRB.

In this talk, I will present a novel, Mutual Information (MI)-based method as an alternative tool for identifying RRB. The proposed method is theoretically justified within the Independent Latent Class Item Response Theory (ILC-IRT) framework for response engagement. However, I begin by briefly presenting the background for RRB, conceptualizing RRB with respect to the observed response time distribution, and introducing the existing RTTMs for identifying RRB. Second, I present the ILC-IRT framework for response engagement.

Finally, I bring MI into the educational and psychological measurement context based on the ILC-IRT framework to define the MI of response Correctness and Time (MICT), a measure of mutual dependency between responses and log RT indicators, define MICT’s empirical counterpart in the realized level for observed samples, and present the proposed MI-based method for identifying RRB. I will also illustrate how the MaxMI method is used and compare it to existing methods, both at the population and realized level.