Analysis validation has been neglected in the Age of Reproducibility

Kathleen E. Lotterhos, Jason H. Moore, Ann E. Stapleton
2018 PLoS Biology  
Increasingly complex statistical models are being used for the analysis of biological data. Recent commentary has focused on the ability to compute the same outcome for a given dataset (reproducibility). We argue that a reproducible statistical analysis is not necessarily valid because of unique patterns of nonindependence in every biological dataset. We advocate that analyses should be evaluated with known-truth simulations that capture biological reality, a process we call "analysis
more » ... ." We review the process of validation and suggest criteria that a validation project should meet. We find that different fields of science have historically failed to meet all criteria, and we suggest ways to implement meaningful validation in training and practice.
doi:10.1371/journal.pbio.3000070 pmid:30532167 pmcid:PMC6301703 fatcat:mh3lk3sdzfcfpkjtl2af57aldm