Filters








1 Hit in 2.2 sec

Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networks [article]

Shawn W. M. Li
2022
One particular finding is that a hierarchically large DNN would have a large reservoir of adaptive symmetries, and when the information capacity of the reservoir exceeds the complexity of the dataset,  ...  The model is analyzed with a method referred as the statistical assembly method that analyzes the coarse-grained behaviors (over a symmetry group) of the heterogeneous hierarchical many-body interaction  ...  Storkey, On the relation between the sharpest direc- tions of Neural Networks with ReLU Activation, in NIPS (2017) DNN loss and the SGD step length, in ICLR (2019).  ... 
doi:10.48550/arxiv.2201.07934 fatcat:7lir4oz3rzcy5erjx4g5qjv3xy