Information in Infinite Ensembles of Infinitely-Wide Neural Networks [article]

Ravid Shwartz-Ziv, Alexander A. Alemi
2019 arXiv   pre-print
In this preliminary work, we study the generalization properties of infinite ensembles of infinitely-wide neural networks. Amazingly, this model family admits tractable calculations for many information-theoretic quantities. We report analytical and empirical investigations in the search for signals that correlate with generalization.
arXiv:1911.09189v2 fatcat:x2o3illcg5fghkqpqodeuf4or4