External correspondence: Decompositions of the mean probability score

J.Frank Yates
<span title="">1982</span> <i title="Elsevier BV"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/g7sdnukm2zc6nahxvhicolp3vu" style="color: black;">Organizational Behavior and Human Performance</a> </i> &nbsp;
Two evaluative criteria for probabilistic forecasting performance, consistency with the axioms of probability theory and external correspondence with the events that ultimately occur, are distinguished. The mean probability, or Brier score (PS), is the scoring rule most commonly used to quantify external correspondence. A review is made of methods for decomposing PS into components that represent distinct and important aspects of external correspondence. Data from an empirical study of
more &raquo; ... ng performance are used to illustrate the interpretation of the components of the most recent decomposition of PS (J. F. Yates, Forecasting performance: A covariance decomposition of the mean probability score. Paper presented at 22nd Annual Meeting of the Psychonomic Society, Philadelphia, November 1981; also an unpublished manuscript). Substantively, the most important finding of the study was a "collapsing" tendency in forecasting behavior, whereby subjects were inclined to report forecasts of .5 when they felt they knew little about the event in question. This finding is problematic because self-reported knowledge was only minimally related to the actual external correspondence of the subjects' forecasts. A survey of uses of PS decompositions suggests, among other things, that current research typically emphasizes calibration, perhaps to the neglect of other, more important dimensions of external correspondence. INTERNAL CONSISTENCY VS EXTERNAL CORRESPONDENCE Probabilistic forecasts are essentially statements by the forecaster of the degree to which he or she is certain that a particular event will occur The assistance of Halimah Hassan and Bruce Carlson in the collecting and coding of data in the reported research is gratefully acknowledged. Requests for reprints should be sent to
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/0030-5073(82)90237-9">doi:10.1016/0030-5073(82)90237-9</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/qohqachdi5c2bm73j5zzskkvbe">fatcat:qohqachdi5c2bm73j5zzskkvbe</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170809024221/https://deepblue.lib.umich.edu/bitstream/handle/2027.42/23907/0000150.pdf?sequence=1" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/29/82/29824502655dcf36dd25babe45423cb374e6c821.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/0030-5073(82)90237-9"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> elsevier.com </button> </a>