Filters








2,284 Hits in 2.6 sec

Compressing Deep Neural Networks using a Rank-Constrained Topology

Preetum Nakkiran, Raziel Alvarez, Rohit Prabhavalkar, Carolina Parada
unpublished
We present a general approach to reduce the size of feed-forward deep neural networks (DNNs).  ...  We propose a rank-constrained topology, which factors the weights in the input layer of the DNN in terms of a low-rank representation: unlike previous work, our technique is applied at the level of the  ...  We propose a scheme to compress an existing fully-trained DNN using a low-rank approximation of the weights associated with individual nodes in the first hidden layer by means of a rank-constrained DNN  ... 
fatcat:ixrgg3vlonb6rffobhqgqlxhzu

Compressing deep neural networks using a rank-constrained topology

Preetum Nakkiran, Raziel Alvarez, Rohit Prabhavalkar, Carolina Parada
2015 Interspeech 2015   unpublished
We present a general approach to reduce the size of feedforward deep neural networks (DNNs).  ...  We propose a rankconstrained topology, which factors the weights in the input layer of the DNN in terms of a low-rank representation: unlike previous work, our technique is applied at the level of the  ...  We propose a scheme to compress an existing fully-trained DNN using a low-rank approximation of the weights associated with individual nodes in the first hidden layer by means of a rank-constrained DNN  ... 
doi:10.21437/interspeech.2015-351 fatcat:t6dbybgzirhynngkspr723n6xi

Literature Review of Deep Network Compression

Ali Alqahtani, Xianghua Xie, Mark W. Jones
2021 Informatics  
In this paper, we present an overview of popular methods and review recent works on compressing and accelerating deep neural networks.  ...  Deep networks often possess a vast number of parameters, and their significant redundancy in parameterization has become a widely-recognized property.  ...  This procedure has been shown to be effective in learning the low-rank constrained networks.  ... 
doi:10.3390/informatics8040077 fatcat:u2dzzibapnf2dbjdqkvgl3pztu

GNN-RL Compression: Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning [article]

Sixing Yu, Arya Mazaheri, Ali Jannesari
2021 arXiv   pre-print
Model compression is an essential technique for deploying deep neural networks (DNNs) on power and memory-constrained resources.  ...  In this paper, we propose a novel multi-stage graph embedding technique based on graph neural networks (GNNs) to identify the DNNs' topology and use reinforcement learning (RL) to find a suitable compression  ...  Conclusion This paper proposed a network compression approach called GNN-RL, which utilizes a graph neural network and a reinforcement learning agent to exploit a topology-aware compression policy.  ... 
arXiv:2102.03214v1 fatcat:dls3bhpvlfd3rlmh45emmvtue4

Overview of the Neural Network Compression and Representation (NNR) Standard

Heiner Kirchhoffer, Paul Haase, Wojciech Samek, Karsten Muller, Hamed Rezazadegan-Tavakoli, Francesco Cricri, Emre Aksu, Miska M. Hannuksela, Wei Jiang, Wei Wang, Shan Liu, Swayambhoo Jain (+3 others)
2021 IEEE transactions on circuits and systems for video technology (Print)  
The standard is designed as a toolbox of compression methods, which can be used to create coding pipelines.  ...  Abstract-Neural Network Coding and Representation (NNR) is the first international standard for efficient compression of neural networks (NNs).  ...  By utilizing a reference signaling mechanism, different elements and components of a neural network can be compressed, carried in the NNR bitstream and then linked to the neural network topology.  ... 
doi:10.1109/tcsvt.2021.3095970 fatcat:wl4tvtpjaveuxng6fqu4tost5y

2020 Index IEEE Transactions on Circuits and Systems for Video Technology Vol. 30

2020 IEEE transactions on circuits and systems for video technology (Print)  
., see Sepas-Moghaddam, A., TCSVT Dec. 2020 4496-4512 Hassanpour, H., see Khosravi, M.H., TCSVT Jan. 2020 48-58 Hatzinakos, D., see 2900-2916 Hayat, M., see 2900-2916 He, C., Hu, Y., Chen, Y., Fan  ...  Blind Image Quality Assessment Using a Deep Bilinear Convolutional Neural Network.  ...  ., +, TCSVT Oct. 2020 3714-3726 A Blind Image Quality Assessment Using a Deep Bilinear Convolutional Neural Network.  ... 
doi:10.1109/tcsvt.2020.3043861 fatcat:s6z4wzp45vfflphgfcxh6x7npu

2020 Index IEEE Transactions on Image Processing Vol. 29

2020 IEEE Transactions on Image Processing  
., +, TIP 2020 3638-3651 A Wave-Shaped Deep Neural Network for Smoke Density Estimation.  ...  ., +, TIP 2020 9678-9688 A Wave-Shaped Deep Neural Network for Smoke Density Estimation.  ... 
doi:10.1109/tip.2020.3046056 fatcat:24m6k2elprf2nfmucbjzhvzk3m

Sparse Neural Networks Topologies [article]

Alfred Bourely, John Patrick Boueri, Krzysztof Choromonski
2017 arXiv   pre-print
Previous approaches using sparcity are all based on fully connected neural network models and create sparcity during training phase, instead we explicitly define a sparse architectures of connections before  ...  As we show, even more compact topologies of the so-called SNN (Sparse Neural Network) can be achieved with the use of structured graphs of connections between consecutive layers of neurons.  ...  Good quality deep neural network/energy-based models might be obtained by imposing sparsity restrictions with nonzero activations [19] , [20] , [21] or using a linear rectifier as a nonlinear mapping  ... 
arXiv:1706.05683v1 fatcat:ehnqw5rx7rewvi2pe36oox4vum

Improving efficiency in convolutional neural networks with multilinear filters

Dat Thanh Tran, Alexandros Iosifidis, Moncef Gabbouj
2018 Neural Networks  
The excellent performance of deep neural networks has enabled us to solve several automatization problems, opening an era of autonomous devices.  ...  Instead of compressing a pre-trained network, in this work, we propose a generic neural network layer structure employing multilinear projection as the primary feature extractor.  ...  The first incorporation of low rank assumption in neural network compression was proposed in [12] , [20] , [11] .  ... 
doi:10.1016/j.neunet.2018.05.017 pmid:29920430 fatcat:gnvv5ldcdndslggu74uzbitkcu

Constrained Optimization Based Low-Rank Approximation of Deep Neural Networks [chapter]

Chong Li, C. J. Richard Shi
2018 Lecture Notes in Computer Science  
used in low-rank approximation.  ...  We present COBLA-Constrained Optimization Based Lowrank Approximation-a systematic method of finding an optimal lowrank approximation of a trained convolutional neural network, subject to constraints in  ...  Our proposed method, named COBLA (Constrained Optimization Based Low-rank Approximation), combines the well-studied low-rank approximation technique in deep neural networks [27, 30, 13, 21, 1, 9, 11,  ... 
doi:10.1007/978-3-030-01249-6_45 fatcat:xcxwdp2pnfah5pkewidmeccx7i

Characterising Across-Stack Optimisations for Deep Convolutional Neural Networks

Jack Turner, Jose Cano, Valentin Radu, Elliot J. Crowley, Michael OrBoyle, Amos Storkey
2018 2018 IEEE International Symposium on Workload Characterization (IISWC)  
In this paper we unify the two viewpoints in a Deep Learning Inference Stack and take an across-stack approach by implementing and evaluating the most common neural network compression techniques (weight  ...  Convolutional Neural Networks (CNNs) are extremely computationally demanding, presenting a large barrier to their deployment on resource-constrained devices.  ...  A. Neural Networks Models We consider the workloads of three CNNs that represent three separate classes of network topology.  ... 
doi:10.1109/iiswc.2018.8573503 dblp:conf/iiswc/TurnerCRCOS18 fatcat:hxxhuovm6fhyhheg55vtwyvsoi

Characterising Across-Stack Optimisations for Deep Convolutional Neural Networks [article]

Jack Turner, José Cano, Valentin Radu, Elliot J. Crowley, Michael O'Boyle, Amos Storkey
2018 arXiv   pre-print
In this paper we unify the two viewpoints in a Deep Learning Inference Stack and take an across-stack approach by implementing and evaluating the most common neural network compression techniques (weight  ...  Convolutional Neural Networks (CNNs) are extremely computationally demanding, presenting a large barrier to their deployment on resource-constrained devices.  ...  A. Neural Networks Models We consider the workloads of three CNNs that represent three separate classes of network topology.  ... 
arXiv:1809.07196v1 fatcat:wxevr5hprveiro5lg2aie5nnem

A Survey of Model Compression and Acceleration for Deep Neural Networks [article]

Yu Cheng, Duo Wang, Pan Zhou, Tao Zhang
2020 arXiv   pre-print
Therefore, a natural thought is to perform model compression and acceleration in deep networks without significantly decreasing the model performance.  ...  Deep neural networks (DNNs) have recently achieved great success in many visual recognition tasks.  ...  In [61] , dynamic deep neural networks (D2NN) were introduced, which were a type of feed-forward deep neural network that selected and executed a subset of D2NN neurons based on the input.  ... 
arXiv:1710.09282v9 fatcat:frwedew2gfe3rjif5ds75jqay4

Trends and Advancements in Deep Neural Network Communication [article]

Felix Sattler, Thomas Wiegand, Wojciech Samek
2020 arXiv   pre-print
To address the challenges of these distributed environments, a wide range of training and evaluation schemes have been developed, which require the communication of neural network parametrizations.  ...  Due to their great performance and scalability properties neural networks have become ubiquitous building blocks of many applications.  ...  Not only are today's algorithms used to enhance the design and management of networks and communication components [33] , ML models such as deep neural networks themselves are being communicated more  ... 
arXiv:2003.03320v1 fatcat:tgs7b6nbovflngyuxhxwkmxhv4

Compact Speaker Embedding: lrx-vector

Munir Georges, Jonathan Huang, Tobias Bocklet
2020 arXiv   pre-print
Deep neural networks (DNN) have recently been widely used in speaker recognition systems, achieving state-of-the-art performance on various benchmarks.  ...  In this paper, we present the lrx-vector system, which is the low-rank factorized version of the x-vector embedding network.  ...  [11] use lowrank projection layers in Recurrent Neural Networks (RNN) for speech recognition. A rank constrained DNN topology for key word spotting is proposed by Nakkiran et al. [12] .  ... 
arXiv:2008.05011v1 fatcat:ns3gaqschne2phcspgoum3qfp4
« Previous Showing results 1 — 15 out of 2,284 results