Deep Learning, Explained: Fundamentals, Explainability, and Bridgeability to Process-based Modelling

Saman Razavi
2021 Environmental Modelling & Software  
7 Recent breakthroughs in artificial intelligence (AI), and particularly in deep learning (DL), have created 8 tremendous excitement and opportunities in the earth and environmental sciences communities. To 9 leverage these new 'data-driven' technologies, however, one needs to understand the fundamental 10 concepts that give rise to DL and how they differ from 'process-based', mechanistic modelling. This paper 11 revisits those fundamentals and addresses 10 questions often posed by earth and
more » ... ironmental 12 scientists with the aid of a real-world modelling experiment. The overarching objective is to contribute to 13 a future of AI-assisted earth and environmental sciences where DL models can (1) embrace the typically 14 ignored knowledge base available, (2) function credibly in 'true' out-of-sample prediction, and (3) handle 15 non-stationarity in earth and environmental systems. Comparing and contrasting earth and 16 environmental problems with prominent AI applications, such as playing chess and trading in stock 17 markets, provides critical insights for better directing future research in this field. 18 Plain Language Summary 19 The recent unprecedented performance of deep learning (DL) in image and language processing has 20 accelerated applications in non-native areas such as earth and environmental sciences where knowledge-21 driven, process-based modelling has dominated to date. A major challenge, however, is DL and process-22 based modelling are rooted in different worldviews towards problem solving. This paper explains the 23 'whats' and 'whys' of DL from first principles and how they are different from those of process-based 24 modelling. A hydrologic modelling experiment is presented to illustrate the fundamental differences 25 between the two worldviews, and to shed light on some critical, but often ignored, issues DL may face in 26 practice. These issues largely arise from the fact that earth and environmental systems are complex, with 27 behaviors that can change in ways that are physically explainable but not seen in the period of record, 28 due to factors such as climate change and human interventions. Such issues must be addressed at the 29 heart of the endeavor to extend DL techniques that embrace the knowledge base available, in anticipation 30 of breakthroughs in an age of big data and computational power. 31 Keywords 32 Artificial intelligence, machine learning, deep learning, artificial neural networks, process-based 33 modelling, earth systems, hydrology 34 35 36 J o u r n a l P r e -p r o o f 2 Key Points 37  DL is rooted in connectionism, hyper-flexibility, and vigorous optimization, which are alien to 38 conventional knowledge-based modelling. 39  A knowledge base is essential to enable credible predictions of complex, open, partially observable, 40 and non-stationary systems. 41  Bridging DL and earth and environmental sciences is still embryonic but has great potential in an age 42 of big data and computational power. 43
doi:10.1016/j.envsoft.2021.105159 fatcat:he5p53ahtveopedzc4monz27wi