Assessing public forecasts to encourage accountability: The case of MIT's Technology Review

Jeffrey Funk, Yih-Kuen Jan
2017 PLoS ONE  
Although high degrees of reliability have been found for many types of forecasts purportedly due to the existence of accountability, public forecasts of technology are rarely assessed and continue to have a poor reputation. This paper's analysis of forecasts made by MIT's Technology Review provides a rare assessment and thus a means to encourage accountability. It first shows that few of the predicted "breakthrough technologies" currently have large markets. Only four have sales greater than
more » ... billion while eight technologies not predicted by Technology Review have sales greater than $10 billion including three with greater than $100 billion and one other with greater than $50 billion. Second, possible reasons for these poor forecasts are then discussed including an over emphasis on the science-based process of technology change, sometimes called the linear model of innovation. Third, this paper describes a different model of technology change, one that is widely used by private companies and that explains the emergence of those technologies that have greater than $10 billion in sales. Fourth, technology change and forecasts are discussed in terms of cognitive biases and mental models. PLOS ONE | https://doi.org/10.1371/journal.pone.product introductions where choices about specific technologies imply forecasts about them [11][12]. Investors reward companies that make good choices, i.e., forecasts, about new technologies. Apple's high market capitalization reflects its successful introduction of MP3 players, smart phones, and tablet computers, all of which imply successful forecasts. Similar conclusions can be drawn from Amazon's entry into eBooks, eReaders, and cloud computing, Facebook's entry into social networking, video advertisements, and smart phone apps, and the evolution of IBM's product lines over the last 100 years. Furthermore, some managers make better forecasts than do other managers because they can "look forward and reason back" [13] in order to identify new technologies and develop better strategies for them. David Yoffie and Michael Cusumano [13] concluded that doing this is a critical skill for managers such as Bill Gates, Steve Jobs, and Andy Grove. Throughout their careers, these three managers could look forward and identify trends, including necessary changes in products and services, and then reason back to develop appropriate strategies for the products and services. On the other hand, public forecasts of technology have been heavily criticized [14][15] [16] with some exceptions [17], in spite of their clear importance. Decision makers in governments, universities, and other public organizations need access to good forecasts to make good decisions about new technologies. Governments must allocate funds to R&D [11][12], universities must choose which technologies to pursue, and cities must determine how to proceed with new infrastructure such as smart cities. This suggests that better technology forecasting is needed and a first step is an assessment of existing forecasts to provide more accountability, to understand the cognitive biases associated with public technology forecasts, and to propose better ways to make forecasts. Cognitive biases are a major reason for poor forecasts and poor decision making in general. People have biases partly because they use heuristics to deal with a complicated world [18] and they are often over confident and miscalibrate their degree of confidence [19] . For example, people assess the relative importance of issues, including new technologies, by the ease of retrieving them from memory [20] , this causes them to be optimistic about technologies that are regularly discussed by their peers or the mass media. Others may be biased towards a single factor and thus focus too much on this factor, rather than consider a wide variety of factors when forecasting technology or something else [1]. These biases often become stronger in groups in which there is strong pressure for conformity [15] [16] [21] . This paper assesses predictions of breakthrough technologies made by MIT's Technology Review between 2001 and 2005 and it explains the (poor) forecasts in terms of cognitive biases particularly those associated with technology change. The next section summarizes the sources and methods of analyzing the data. The third section presents the current sizes of the markets for the technologies predicted and missed by MIT's Technology Review. The fourth section interprets these results and possible reasons for the poor forecasts including an over emphasis on the science-based process of technology change, sometimes called the linear model of innovation. The over emphasis on the science-based process of technology change is a form of cognitive bias that comes from an emphasis on science in universities. The fifth section discusses an alternative model of technology change that is widely used by private companies, that explains the successful predictions by Technology Review and those missed by Technology Review, and that is consistent with one relatively successful technology forecast [17] [22] . The sixth section discusses the theoretical and practical implications of searching for commercially viable technologies including cognitive biases, mental models and technological choices, including those for smart cities. Assessing public forecasts to encourage accountability PLOS ONE | https://doi.
doi:10.1371/journal.pone.0183038 pmid:28797114 pmcid:PMC5552300 fatcat:5u4tbyglurcjzckuudhggaqmma