The Worst Performance Rule as Moderation: New Methods for Worst Performance Analysis

Gidon Frischkorn, Anna-Lena Schubert, Andreas Neubauer, Dirk Hagemann
2016 Journal of Intelligence  
Worst performance in cognitive processing tasks shows larger relationships to general intelligence than mean or best performance. This so called Worst Performance Rule (WPR) is of major theoretical interest for the field of intelligence research, especially for research on mental speed. In previous research, the increases in correlations between task performance and general intelligence from best to worst performance were mostly described and not tested statistically. We conceptualized the WPR
more » ... s moderation, since the magnitude of the relation between general intelligence and performance in a cognitive processing task depends on the performance band or percentile of performance. On the one hand, this approach allows testing the WPR for statistical significance and on the other hand, it may simplify the investigation of possible constructs that may influence the WPR. The application of two possible implementations of this approach is shown and compared to results of a traditional worst performance analysis. The results mostly replicate the WPR. Beyond that, a comparison of results on the level of unstandardized relationships (e.g., covariances or unstandardized regression weights) to results on the level of standardized relationships (i.e., correlations) indicates that increases in the inter-individual standard deviation from best to worst performance may play a crucial role for the WPR. Altogether, conceptualizing the WPR as moderation provides a new and straightforward way to conduct Worst Performance Analysis and may help to incorporate the WPR more prominently into empirical practice of intelligence research. of 22 bands or best and worst memory performance were correlated with a measure of general intelligence. The absolute size of these correlations mostly increased from best to worst performance, suggesting that worst performance is more closely related to general intelligence than mean or best performance. This phenomenon is called the worst performance rule (WPR) [4] . Although a few studies replicated this phenomenon [5-10], it has yet to be acknowledged adequately in the field of intelligence research. In previous research, the analysis of the WPR, or so called worst performance analysis (WPA), mostly described increases in correlations between performance in cognitive processing tasks and general intelligence from best to worst performance instead of providing an adequate statistical test for this phenomenon (for a detailed review, see [3] ). The present work conceptualizes the WPR as moderation and introduces new approaches to analyze and test the WPR for statistical significance. These analyses try to overcome the rather descriptive approach of formerly published WPA and offer new possibilities to search for empirical foundations of the WPR. The here presented analysis may hence constitute a useful step towards an accurate test for the WPR that may in turn help to better distinguish between different theoretical explanations for the WPR in future empirical research. The Phenomenon of the Worst Performance Rule The WPR was first explicitly described by Larson and Alderton [4] who showed that correlations of general intelligence with RTs in a simple reaction time paradigm increased from best (r BP = −.20) to worst performance (r WP = −.37). This initiated a number of conceptually associated studies [5] [6] [7] [8] [9] [10] [11] with different related tasks of which all but one [11] reproduced the basic phenomenon of the WPR. Interestingly, the WPR did not only occur in speeded processing tasks but in non-speeded tasks as well [7, 8] . In a multi-trial word recall task, the number of words recalled in worst performance trials correlated more strongly with general intelligence (r WP = .38) than in best performance trials (r BP = .13). The effect size Cohen's q [12] for this difference in correlations was q = .27. Moreover, even the number of memorizing strategies in worst performance trials showed a higher correlation with general intelligence in worst performance trials (r = .24) than in best performance trials (r = .12, q = .11). However, only minimal recall performance (i.e., worst performance) predicted general intelligence, and strategy use showed no incremental validity [7] . With these results, the WPR questions some of the core assumptions within theories of intelligence [3] . In particular, the WPR shows that it may not be the inter-individual difference in average performance that depicts differences in intelligence best, rather inter-individual differences in worst performance seem to be most predictive for general intelligence. Although the standard deviation of intra-individual reaction time distributions (RTsd) has been discussed as an additional and supposedly more valid predictor for general intelligence (e.g., the oscillation theory [13] [14] [15] ), a recent meta-analysis has shown that mean RT and the RTsd are equally valid predictors for general intelligence [16] . In addition, the magnitude of the WPR relies on the g-loading of a task [3] . This suggests that processes fundamental to the WPR may well be processes fundamental to g [17] . Therefore, the WPR is an interesting phenomenon that needs to be studied further. Theoretically, two main approaches to understand the WPR have been suggested so far. Either worst performance more strongly reflects the speed of information accumulation [18, 19] or worst performance trials occur when a person has lapses in attention resulting in longer reaction times [20, 21] . Both approaches acknowledge that worst performance trials may contain information on processes that are not adequately represented in mean performance. In fact, the phenomenon of the WPR may be one of the possibilities to shed light on interactions between different cognitive processes (e.g., information accumulation and attentional control), because the WPR does not necessarily represent a cognitive process on its own, but it may occur in the interplay of different cognitive processes. Beyond that, some methodological explanations have been suggested that may explain the WPR [3] . Specifically, Coyle [3] discussed five possible methodological explanations: (1) the role of outliers; (2) variance compression in best performance; (3) skewness of the intra-individual performance distribution; (4) differences in measurement reliability between best and worst
doi:10.3390/jintelligence4030009 fatcat:se5zmelwyreepljl4x5sk36g44