Predictive coding is a consequence of energy efficiency in recurrent neural networks [article]

Abdullahi Ali, Nasir Ahmad, Elgar de Groot, Marcel A. J. van Gerven, Tim C. Kietzmann
2021 bioRxiv   pre-print
AbstractPredictive coding represents a promising framework for understanding brain function. It postulates that the brain continuously inhibits predictable sensory input, ensuring a preferential processing of surprising elements. A central aspect of this view is its hierarchical connectivity, involving recurrent message passing between excitatory bottom-up signals and inhibitory top-down feedback. Here we use computational modelling to demonstrate that such architectural hard-wiring is not
more » ... sary. Rather, predictive coding is shown to emerge as a consequence of energy efficiency. When training recurrent neural networks to minimise their energy consumption while operating in predictive environments, the networks self-organise into prediction and error units with appropriate inhibitory and excitatory interconnections, and learn to inhibit predictable sensory input. Moving beyond the view of purely top-down driven predictions, we demonstrate via virtual lesioning experiments that networks perform predictions on two timescales: fast lateral predictions among sensory units, and slower prediction cycles that integrate evidence over time.
doi:10.1101/2021.02.16.430904 fatcat:t6wqlz7uabbdfd2qjdcaeomu4q