CrescendoNet: A New Deep Convolutional Neural Network with Ensemble Behavior

Xiang Zhang, Nishant Vishwamitra, Hongxin Hu, Feng Luo
2018 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)  
We introduce a new deep convolutional neural network, CrescendoNet, by stacking simple building blocks without residual connections. Each Crescendo block contains independent convolution paths with increased depths. The numbers of convolution layers and parameters are only increased linearly in Crescendo blocks. In experiments, CrescendoNet with only 15 layers outperforms almost all networks without residual connections on benchmark datasets, CIFAR10, CIFAR100, and SVHN. Given sufficient amount
more » ... of data as in SVHN dataset, CrescendoNet with 15 layers and 4.1M parameters can match the performance of DenseNet-BC with 250 layers and 15.3M parameters. CrescendoNet provides a new way to construct high performance deep convolutional neural networks with simple network architecture. Moreover, through investigating the behavior and performance of subnetworks in CrescendoNet, we note that the high performance of CrescendoNet may come from its implicit ensemble behavior. Furthermore, the independence between paths in CrescendoNet allows us to introduce a new path-wise training procedure, which can reduce the memory needed for training.
doi:10.1109/icmla.2018.00053 dblp:conf/icmla/ZhangVH018 fatcat:3imfobuzjveellpwoq5qnpoudm