7,682 Hits in 4.7 sec

Clustering Convolutional Kernels to Compress Deep Neural Networks [chapter]

Sanghyun Son, Seungjun Nah, Kyoung Mu Lee
2018 Lecture Notes in Computer Science  
In this paper, we propose a novel method to compress CNNs by reconstructing the network from a small set of spatial convolution kernels.  ...  Kernels in the same cluster share their weights, and we fine-tune the model while keeping the compressed state.  ...  Related works Network quantization is one of the typical approaches to compress deep neural networks. It focuses on reducing the number of bits to represent each parameter.  ... 
doi:10.1007/978-3-030-01237-3_14 fatcat:7ky4namhfrerffnwfurl357wqa

Coarse and fine-grained automatic cropping deep convolutional neural network [article]

Jingfei Chang
2020 arXiv   pre-print
First, cluster the intermediate feature maps of the convolutional neural network to obtain the network structure after coarse-grained clipping, and then use the particle swarm optimization algorithm to  ...  This paper proposes a coarse and fine-grained automatic pruning algorithm, which can achieve more efficient and accurate compression acceleration for convolutional neural networks.  ...  algorithm to delete the redundant feature maps and their corresponding convolution kernels in each layer to realize the coarse-grained clipping of the convolutional neural network.  ... 
arXiv:2010.06379v2 fatcat:b4rgyq3kpzbipngjmzbpuqkxni

Slim and Efficient Neural Network Design for Resource-Constrained SAR Target Recognition

Hongyi Chen, Fan Zhang, Bo Tang, Qiang Yin, Xian Sun
2018 Remote Sensing  
Deep convolutional neural networks (CNN) have been recently applied to synthetic aperture radar (SAR) for automatic target recognition (ATR) and have achieved state-of-the-art results with significantly  ...  Experimental results show that the convolutional neural networks for SAR-ATR can be compressed by 40 × without loss of accuracy, and the number of multiplication can be reduced by 15 × .  ...  Due to the deep neural network model and large amount of convolution calculations, it is difficult to deploy the ATR networks on mobile platform with limited hardware resources.  ... 
doi:10.3390/rs10101618 fatcat:lfrmlwujarhqpds3nj65zozcri

Information Bottleneck Theory on Convolutional Neural Networks [article]

Junjie Li, Ding Liu
2021 arXiv   pre-print
Recent years, many researches attempt to open the black box of deep neural networks and propose a various of theories to understand it.  ...  In this paper, we employ IB theory to understand the dynamic behavior of convolutional neural networks (CNNs) and investigate how the fundamental features such as convolutional layer width, kernel size  ...  Furthermore, it shows us the extremely deep neural network is probably not the right way to do deep learning. 4 .  ... 
arXiv:1911.03722v2 fatcat:pnb5mwnudfad7fma2z27syg33m

Compressing Deep Networks by Neuron Agglomerative Clustering

Li-Na Wang, Wenxue Liu, Xiang Liu, Guoqiang Zhong, Partha Pratim Roy, Junyu Dong, Kaizhu Huang
2020 Sensors  
However, high-performance deep architectures are often accompanied by a large storage space and long computational time, which make it difficult to fully exploit many deep neural networks (DNNs), especially  ...  In this paper, to tackle this problem, we introduce a method for compressing the structure and parameters of DNNs based on neuron agglomerative clustering (NAC).  ...  Acknowledgments: The authors would like to thank the guest editors and the anonymous reviewers for their work and time on the publication of this paper.  ... 
doi:10.3390/s20216033 pmid:33114078 pmcid:PMC7660330 fatcat:hetsfskkifdlbhdgkbdh66ked4

Finding Storage- and Compute-Efficient Convolutional Neural Networks

Daniel Becking, Simon Wiedemann, Klaus-Robert Müller
2020 Zenodo  
Convolutional neural networks (CNNs) have taken the spotlight in a variety of machine learning applications.  ...  More precisely, the framework yields sparse and ternary neural networks, i.e. networks with many parameters set to zero and the non-zero parameters quantized from 32 bit to 2 bit.  ...  Deep neural networks (DNNs) are considered as "deep" when the number of layers is large.  ... 
doi:10.5281/zenodo.5501151 fatcat:zjh4kngadrgtdgzniphqrvfndq

2020 Index IEEE Journal of Selected Topics in Signal Processing Vol. 14

2020 IEEE Journal on Selected Topics in Signal Processing  
., +, JSTSP Jan. 2020 5-26 Symmetrick-Means for Deep Neural Network Compression and Hardware Acceleration on FPGAs. Jain, A., +, JSTSP May 2020 737-749 Universal Deep Neural Network Compression.  ...  ., +, JSTSP May 2020 727-736 Binary Outer Product Expansion of Convolutional Kernels. Sun, Z.H., JSTSP May 2020 871-883 DeepCABAC: A Universal Compression Algorithm for Deep Neural Net- works.  ... 
doi:10.1109/jstsp.2020.3029672 fatcat:6twwzcqpwzg4ddcu2et75po77u

Editorial: Special Issue on Compact Deep Neural Networks With Industrial Applications

Lixin Fan, Diana Marculescu, Werner Bailer, Yurong Chen
2020 IEEE Journal on Selected Topics in Signal Processing  
Compressed DNNs and Deep Compressive Sensing One of the earliest methods proposed for reducing the computational cost of deep neural networks is compression.  ...  In "Discriminative Layer Pruning for Convolutional Neural Networks" Jordao et al. propose removing entire convolutional layers to reduce network depth.  ... 
doi:10.1109/jstsp.2020.3006323 fatcat:d75ni7ocajb4pemovq2l3ton4i

Compressing Neural Networks With Inter Prediction and Linear Transformation

Kang-Ho Lee, Sung-Ho Bae
2021 IEEE Access  
Deep Compression, a pruning-quantization framework, became a milestone in model compression research of deep neural networks.  ...  INTRODUCTION R ECENTLY, Deep Neural Networks (DNN), specifically Convolutional Neural Networks (CNN), are showing exceptional performance compared with traditional methods for a wide variety of tasks in  ... 
doi:10.1109/access.2021.3077596 fatcat:smb4ig3hgzds5ff2w4e3ic2wee

CNN inference acceleration using dictionary of centroids [article]

D.Babin, I.Mazurenko, D.Parkhomenko, A.Voloshko
2018 arXiv   pre-print
In this article we present a flexible method to decrease both computational complexity of convolutional layers in inference as well as amount of space to store them.  ...  It is well known that multiplication operations in convolutional layers of common CNNs consume a lot of time during inference stage.  ...  Here we enclose method to accelerate and compress convolutional layers of neural networks. Method based on convolution quantization followed by clustering.  ... 
arXiv:1810.08612v1 fatcat:lbnrtdkwgbe3vn6ana2d3iqpjm

Compressing deep neural networks by matrix product operators [article]

Ze-Feng Gao, Song Cheng, Rong-Qiang He, Z. Y. Xie, Hui-Hai Zhao, Zhong-Yi Lu, Tao Xiang
2019 arXiv   pre-print
Compressing a deep neural network to reduce its number of variational parameters but not its prediction power is an important but challenging problem towards the establishment of an optimized scheme in  ...  A deep neural network is a parameterization of a multi-layer mapping of signals in terms of many alternatively arranged linear and nonlinear transformations.  ...  In this work, we have proposed to use MPO to compress the transformation matrices in deep neural networks.  ... 
arXiv:1904.06194v1 fatcat:lpmrv7lvpjbcbdsdsotc4vhtby

Compressed Deep Learning to Classify Arrhythmia in an Embedded Wearable Device

Kwang-Sig Lee, Hyun-Joon Park, Ji Eon Kim, Hee Jung Kim, Sangil Chon, Sangkyu Kim, Jaesung Jang, Jin-Kook Kim, Seongbin Jang, Yeongjoon Gil, Ho Sung Son
2022 Sensors  
For this reason, this study developed efficient deep learning with model compression, which is designed to use ECG data and classify arrhythmia in an embedded wearable device.  ...  The weight size of the compressed model registered a remarkable decrease from 743 MB to 76 KB (1/10000), whereas its performance was almost the same as its original counterpart.  ...  The convolutional neural network has convolutional layers, in which a kernel passes across input data and performs "convolution", that is, computes the dot product of its own elements and their input-data  ... 
doi:10.3390/s22051776 pmid:35270923 pmcid:PMC8914813 fatcat:tpbdea4kd5cq7k3gpcftlbddna

Quantisation and Pruning for Neural Network Compression and Regularisation [article]

Kimessha Paupamah, Steven James, Richard Klein
2020 arXiv   pre-print
Deep neural networks are typically too computationally expensive to run in real-time on consumer-grade hardware and low-powered devices.  ...  Our results show that pruning and quantisation compresses these networks to less than half their original size and improves their efficiency, particularly on MobileNet with a 7x speedup.  ...  Convolutional Neural Networks Convolutional neural networks are similar to feedforward neural networks, except their layers are composed of convolutional layers which have a height, width and depth.  ... 
arXiv:2001.04850v1 fatcat:qttpea4ppjavhcyfnblsn6q6h4

Unifying and Merging Well-trained Deep Neural Networks for Inference Stage

Yi-Min Chou, Yi-Ming Chan, Jia-Hong Lee, Chih-Yi Chiu, Chu-Song Chen
2018 Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence  
We propose a novel method to merge convolutional neural-nets for the inference stage.  ...  As it preserves the general architectures and leverages the co-used weights of well-trained networks, a substantial training overhead can be reduced to shorten the system development time.  ...  Introduction The research on deep neural networks has gotten a rapid progress and achievement recently.  ... 
doi:10.24963/ijcai.2018/283 dblp:conf/ijcai/ChouCLCC18 fatcat:or2roofsjzdxpmbmz4gucvks3y

Cramnet: Layer-wise Deep Neural Network Compression with Knowledge Transfer from a Teacher Network [article]

Jon Hoffman
2019 arXiv   pre-print
This thesis develops a method that can compress networks to less than 10% of memory and less than 25% of computational power, without loss of accuracy, and without creating sparse networks that require  ...  Previous studies have shown that neural networks have vastly more connections than they actually need to do their work.  ...  ACKNOWLEDGEMENTS I would like to thank everyone, especially Melanie and my parents who supported me. I also thank my employer, L3 Aeromet, which paid for my education.  ... 
arXiv:1904.05982v1 fatcat:mzbgwrx3pnaure4mecujnioqwa
« Previous Showing results 1 — 15 out of 7,682 results