Why we (usually) don't have to worry about multiple comparisons

Andrew E. Gelman, Jennifer Hill, Masanao Yajima
2017
Multiple comparisons using multilevel models What is the multiple comparisons problem? Why don't I (usually) care? Some stories Statistical framework and multilevel modeling SAT coaching in 8 schools Effects of electromagnetic fields at 38 frequencies Teacher and school effects in NYC schools Grades and classroom seating Beautiful parents have more daughters Comparing test scores across states Teacher and school effects in NYC schools Goal is to estimate range of variation (How important are
more » ... ow important are teachers? Schools?) Key statistic is year-to-year persistence (e.g., for teachers ranked in top 25% one year, how well do they do the next?) The "multiple comparisons" issue never arises! Multiple comparisons using multilevel models What is the multiple comparisons problem? Why don't I (usually) care? Some stories Statistical framework and multilevel modeling SAT coaching in 8 schools Effects of electromagnetic fields at 38 frequencies Teacher and school effects in NYC schools Grades and classroom seating Beautiful parents have more daughters Comparing test scores across states Teacher and school effects in NYC schools Goal is to estimate range of variation (How important are teachers? Schools?) Key statistic is year-to-year persistence (e.g., for teachers ranked in top 25% one year, how well do they do the next?) The "multiple comparisons" issue never arises! Multiple comparisons using multilevel models What is the multiple comparisons problem? Why don't I (usually) care? Some stories Statistical framework and multilevel modeling SAT coaching in 8 schools Effects of electromagnetic fields at 38 frequencies Teacher and school effects in NYC schools Grades and classroom seating Beautiful parents have more daughters Comparing test scores across states Teacher and school effects in NYC schools Goal is to estimate range of variation (How important are teachers? Schools?) Key statistic is year-to-year persistence (e.g., for teachers ranked in top 25% one year, how well do they do the next?) The "multiple comparisons" issue never arises! Multiple comparisons using multilevel models What is the multiple comparisons problem? Why don't I (usually) care? Some stories Statistical framework and multilevel modeling SAT coaching in 8 schools Effects of electromagnetic fields at 38 frequencies Teacher and school effects in NYC schools Grades and classroom seating Beautiful parents have more daughters Comparing test scores across states Teacher and school effects in NYC schools Goal is to estimate range of variation (How important are teachers? Schools?) Key statistic is year-to-year persistence (e.g., for teachers ranked in top 25% one year, how well do they do the next?) The "multiple comparisons" issue never arises!
doi:10.7916/d8g73mf6 fatcat:sj7af3dqjbatbdzyji62qrbbky