Scoring Probabilistic Forecasts: The Importance of Being Proper

Jochen Bröcker, Leonard A. Smith
2007 Weather and forecasting  
Questions remain regarding how the skill of operational probabilistic forecasts is most usefully evaluated or compared, even though probability forecasts have been a longstanding aim in meteorological forecasting. This paper explains the importance of employing proper scores when selecting beween the various measures of forecast skill. It is demonstrated that only proper scores provide internally consistent evaluations of probability forecasts, justifying the focus on proper scores
more » ... of any attempt to influence the behaviour of a forecaster. Another property of scores, locality, is discussed. Several scores are examined in this light. There is, effectively, only one proper, local score for probability forecasts of a continuous variable. It is also noted that operational needs of weather forecasts suggest that the current concept of a score may be too narrow; a possible generalisation is motivated and discussed in the context of propriety and locality.
doi:10.1175/waf966.1 fatcat:5qan2l5inzgolmdffmddcxbkaq