A Deep Neural Network Architecture Using Dimensionality Reduction with Sparse Matrices [chapter]

Wataru Matsumoto, Manabu Hagiwara, Petros T. Boufounos, Kunihiko Fukushima, Toshisada Mariyama, Zhao Xiongxin
2016 Lecture Notes in Computer Science  
Matsumoto, W.; Hagiwara, M.; Boufounos, P.T.; Fukushima, K.; Mariyama, T.; Xiongxin, Z. Abstract We present a new deep neural network architecture, motivated by sparse random matrix theory that uses a low-complexity embedding through a sparse matrix instead of a conventional stacked autoencoder. We regard autoencoders as an information-preserving dimensionality reduction method, similar to random projections in compressed sensing. Thus, exploiting recent theory on sparse matrices for
more » ... ity reduction, we demonstrate experimentally that classification performance does not deteriorate if the autoencoder is replaced with a computationally-efficient sparse dimensionality reduction matrix. International Conference on Neural Information Processing (ICONIP) This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Abstract. We present a new deep neural network architecture, motivated by sparse random matrix theory that uses a low-complexity embedding through a sparse matrix instead of a conventional stacked autoencoder. We regard autoencoders as an information-preserving dimensionality reduction method, similar to random projections in compressed sensing. Thus, exploiting recent theory on sparse matrices for dimensionality reduction, we demonstrate experimentally that classification performance does not deteriorate if the autoencoder is replaced with a computationally-efficient sparse dimensionality reduction matrix.
doi:10.1007/978-3-319-46681-1_48 fatcat:e3e7fkwkrvgadliavosmsazmka