Using GAMMs to model trial-by-trial fluctuations in experimental data: More risks but hardly any benefit [post]

Rüdiger Thul, Kathy Conklin, Dale J. Barr
2020 unpublished
Data from each subject in a repeated-measures experiment forms a time series, which may include trial-by-trial fluctuations arising from human factors such as practice or fatigue. Concerns about the statistical implications of such effects have increased the popularity of Generalized Additive Mixed Models (GAMMs), a powerful technique for modeling wiggly patterns. We question these statistical concerns and investigate the costs and benefits of using GAMMs relative to linear mixed-effects models
more » ... (LMEMs). In a Monte Carlo simulation study, LMEMs that ignored time-varying effects were no more prone to false positives than GAMMs. Although GAMMs generally boosted power for within-subject effects, they reduced power for between-subject effects, sometimes to a severe degree. Our results signal the importance of proper subject-level randomization as the main defense against statistical artifacts due to by-trial fluctuations.
doi:10.31234/osf.io/ywkeq fatcat:lu6vimj5p5cbhb2h2lnfith35y