Reproducibility of statistical test results based on p-value

Takashi Yanagawa
2020 Japanese Journal of Biometrics  
Reproducibility is the essence of a scientific research. Focusing on two-sample problems we discuss in this paper the reproducibility of statistical test results based on p-values. First, demonstrating large variability of p-values it is shown that p-values lack the reproducibility, in particular, if sample sizes are not enough. Second, a sample size formula is developed to assure the reproducibility probability of p-value at given level by assuming normal distributions with known variance.
more » ... lly, the sample size formula for the reproducibility in general framework is shown equivalent to the sample size formula that has been developed in the Neyman-Pearson type testing statistical hypothesis by employing the level of significance and size of power.
doi:10.5691/jjb.40.69 fatcat:v4pity3xrjablhn5prtag3k42i