Reproducibility is the essence of a scientific research. Focusing on two-sample problems we discuss in this paper the reproducibility of statistical test results based on p-values. First, demonstrating large variability of p-values it is shown that p-values lack the reproducibility, in particular, if sample sizes are not enough. Second, a sample size formula is developed to assure the reproducibility probability of p-value at given level by assuming normal distributions with known variance.

more »

... lly, the sample size formula for the reproducibility in general framework is shown equivalent to the sample size formula that has been developed in the Neyman-Pearson type testing statistical hypothesis by employing the level of significance and size of power.