More Voodoo Correlations: When Average-Based Measures Inflate Correlations

Andrew Brand, Michael T. Bradley
2012 The Journal of general psychology  
A Monte-Carlo simulation was conducted to assess the extent that a correlation estimate can be inflated when an average-based measure is used in a commonly employed correlational design. The results from the simulation reveal that the inflation of the correlation estimate can be substantial, up to 76%. Additionally, data was re-analyzed from two previously published studies to determine the extent that the correlation estimate was inflated due to the use of an averaged based measure. The
more » ... yses reveal that correlation estimates had been inflated by just over 50% in both studies. Although these findings are disconcerting, we are somewhat comforted by the fact that there is a simple and easy analysis that can be employed to prevent the inflation of the correlation estimate that we have simulated and observed. CENTRAL TO THE INTEGRITY OF PSYCHOLOGICAL RESEARCH is the accuracy of its reported effect size estimates. Due to the dominance of Null Hypothesis Significance Testing (NHST), the reporting of the effect size was at times somewhat ignored and its importance de-emphasized. This problem has changed with recommendations by Wilkinson and the APA Task Force on Statistical Inference (1999). These recommendations have encouraged and promoted the reporting of effect sizes. Consequently, the accuracy of observed effect sizes and their interpretability should now be of primary concern to researchers. The emphasis on effect size reporting has resulted in a wider understanding amongst researchers that published effect sizes are likely to be inflated estimates
doi:10.1080/00221309.2012.703711 pmid:24837177 fatcat:hwarxe5qpzap3awz6wgdb5vstu