Filters








718 Hits in 6.5 sec

Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition [article]

Vadim Lebedev, Yaroslav Ganin, Maksim Rakhuba, Ivan Oseledets, Victor Lempitsky
2015 arXiv   pre-print
We propose a simple two-step approach for speeding up convolution layers within large convolutional neural networks based on tensor decomposition and discriminative fine-tuning.  ...  After such replacement, the entire network is fine-tuned on the training data using standard backpropagation process.  ...  METHOD Overall our method is a conceptually simple two-step approach: (1) take a convolutional layer and decompose its kernel using CP-decomposition, (2) fine-tune the entire network using backpropagation  ... 
arXiv:1412.6553v3 fatcat:l4v2bjjmajblvanqravwugwapm

CP-decomposition with Tensor Power Method for Convolutional Neural Networks Compression [article]

Marcella Astrid, Seung-Ik Lee
2017 arXiv   pre-print
Convolutional Neural Networks (CNNs) has shown a great success in many areas including complex image classification tasks.  ...  We also propose an iterative fine tuning, with which we fine-tune the whole network after decomposing each layer, but before decomposing the next layer.  ...  Whole Network Decomposition In this section, we explain the results of decomposing the whole network with CP-TPM and iterative fine tuning.  ... 
arXiv:1701.07148v1 fatcat:3ikj6emzpbfgvafgfaxyypu4ta

Convolutional neural networks compression with low rank and sparse tensor decompositions [article]

Pavel Kaloshin
2020 arXiv   pre-print
In this work, we consider a neural network compression method based on tensor decompositions.  ...  Convolutional neural networks show outstanding results in a variety of computer vision tasks.  ...  Ideally this relation should let us search for the best parameters for each layer in terms of compression or speed up with given accuracy drop without the need to fine-tune on each iteration (which takes  ... 
arXiv:2006.06443v1 fatcat:jervtncmu5bsvittktzabmo6b4

Reducing Parameters of Neural Networks via Recursive Tensor Approximation

Kyuahn Kwon, Jaeyong Chung
2022 Electronics  
This process factorizes a given network, yielding a deeper, less dense, and weight-shared network with good initial weights, which can be fine-tuned by gradient descent.  ...  Large-scale neural networks have attracted much attention for surprising results in various cognitive tasks such as object detection and image classification.  ...  They demonstrated 2× speed-ups of convolutional layers within a 1% accuracy drop. Lebedev et al. [9] used CP decomposition for 4-tensors and fine-tuned the whole network.  ... 
doi:10.3390/electronics11020214 fatcat:jgyhnb4punftbi5qcv4i3ndebu

Iterative Low-Rank Approximation for CNN Compression [article]

Maksym Kholiavchenko
2019 arXiv   pre-print
We propose iterative approach of applying low-rank approximation to compress deep convolutional neural networks.  ...  Deep convolutional neural networks contain tens of millions of parameters, making them impossible to work efficiently on embedded devices.  ...  Decomposition algorithms Convolutional neural network can be comprised of convolutional layers followed by fully connected layers.  ... 
arXiv:1803.08995v2 fatcat:lstgp7l4gngbhgrfg547pec4qu

MUSCO: Multi-Stage Compression of neural networks [article]

Julia Gusak, Maksym Kholiavchenko, Evgeny Ponomarev, Larisa Markeeva, Ivan Oseledets, Andrzej Cichocki
2019 arXiv   pre-print
The low-rank tensor approximation is very promising for the compression of deep neural networks.  ...  We propose a new simple and efficient iterative approach, which alternates low-rank factorization with a smart rank selection and fine-tuning.  ...  By choosing rank in such a way, we can control the speed-up of each convolutional layer.  ... 
arXiv:1903.09973v4 fatcat:o74mrorinbd6lpfrmev2iseg3a

Automated Multi-Stage Compression of Neural Networks

Julia Gusak, Maksym Kholiavchenko, Evgeny Ponomarev, Larisa Markeeva, Philip Blagoveschensky, Andrzej Cichocki, Ivan Oseledets
2019 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)  
Low-rank tensor approximations are very promising for compression of deep neural networks.  ...  We propose a new simple and efficient iterative approach, which alternates lowrank factorization with smart rank selection and fine-tuning.  ...  Our method is designed to compress any neural network architecture with convolutional and fully connected layers using Tucker-2, CP or SVD decomposition with two different strategies of automatic rank  ... 
doi:10.1109/iccvw.2019.00306 dblp:conf/iccvw/GusakKPMBCO19 fatcat:l3uyzfgicjgvnl6bb77br6vdrq

Training Binary Weight Networks via Semi-Binary Decomposition [chapter]

Qinghao Hu, Gang Li, Peisong Wang, Yifan Zhang, Jian Cheng
2018 Lecture Notes in Computer Science  
We also implement binary weight AlexNet on FPGA platform, which shows that our proposed method can achieve ∼ 9× speed-ups while reducing the consumption of on-chip memory and dedicated multipliers significantly  ...  Recently binary weight networks have attracted lots of attentions due to their high computational efficiency and small parameter size.  ...  [25] propose to use Block Term Decomposition to speed up the convolutional layers. The Block Term Decomposition can be regarded as a compromise between CP-decomposition and Tucker decomposition.  ... 
doi:10.1007/978-3-030-01261-8_39 fatcat:emdugucdnjh5vfo4l2blid427a

Extreme Network Compression via Filter Group Approximation [chapter]

Bo Peng, Wenming Tan, Zheyang Li, Shun Zhang, Di Xie, Shiliang Pu
2018 Lecture Notes in Computer Science  
In this paper we propose a novel decomposition method based on filter group approximation, which can significantly reduce the redundancy of deep convolutional neural networks (CNNs) while maintaining the  ...  Besides, experiments demonstrate that our method is conducive to alleviating degeneracy of the compressed network, which hurts the convergence and performance of the network.  ...  Brighter pixel indicates higher correlation 20 . 20 Lebedev, V., Ganin, Y., Rakhuba, M., Oseledets, I., Lempitsky, V.: Speedingup convolutional neural networks using fine-tuned cp-decomposition.  ... 
doi:10.1007/978-3-030-01237-3_19 fatcat:x2mgqglbpjdgndcaairr6ngsui

T-Net: Parametrizing Fully Convolutional Nets with a Single High-Order Tensor [article]

Jean Kossaifi, Adrian Bulat, Georgios Tzimiropoulos, Maja Pantic
2019 arXiv   pre-print
In this paper, we propose to fully parametrize Convolutional Neural Networks (CNNs) with a single high-order, low-rank tensor.  ...  of the network (e.g. number of convolutional blocks, depth, number of stacks, input features, etc).  ...  The authors in [23] propose such parametrization of individual convolutional layers using CP decomposition with the goal of speeding them up.  ... 
arXiv:1904.02698v1 fatcat:s2io24rw2vf4pjj6v5n4ltye3m

Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications [article]

Yong-Deok Kim, Eunhyeok Park, Sungjoo Yoo, Taelim Choi, Lu Yang, Dongjun Shin
2016 arXiv   pre-print
The proposed scheme consists of three steps: (1) rank selection with variational Bayesian matrix factorization, (2) Tucker decomposition on kernel tensor, and (3) fine-tuning to recover accumulated loss  ...  Although the latest high-end smartphone has powerful CPU and GPU, running deeper convolutional neural networks (CNNs) for complex tasks such as ImageNet classification on mobile devices is challenging.  ...  The FFT method was used to speed-up convolution (Mathieu et al., 2013) . In (Vanhoucke et al., 2011) , CPU code optimizations to speed-up the execution of CNN are extensively explored.  ... 
arXiv:1511.06530v2 fatcat:chbndqpxvrfd7ld33p7fnc3vtm

Network Decoupling: From Regular to Depthwise Separable Convolutions [article]

Jianbo Guo, Yuxi Li, Weiyao Lin, Yurong Chen, Jianguo Li
2018 arXiv   pre-print
And then we propose network decoupling (ND), a training-free method to accelerate convolutional neural networks (CNNs) by transferring pre-trained CNN models into the MobileNet-like depthwise separable  ...  We further verify through experiments that the proposed method is orthogonal to other training-free methods like channel decomposition, spatial decomposition, etc.  ...  This verifies network decoupling and channel decomposition/pruning are intrinsically complementary, and we can combine our decoupling with them to speed up CNNs further. [31] 6.52G 4.72G SD [14] 7.20G  ... 
arXiv:1808.05517v1 fatcat:hlw5g6ibwbehbodemwojzqkdyq

Quantized Convolutional Neural Networks for Mobile Devices

Jiaxiang Wu, Cong Leng, Yuhang Wang, Qinghao Hu, Jian Cheng
2016 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)  
Recently, convolutional neural networks (CNN) have demonstrated impressive performance in various computer vision tasks.  ...  Extensive experiments on the ILSVRC-12 benchmark demonstrate 4 ∼ 6× speed-up and 15 ∼ 20× compression with merely one percentage loss of classification accuracy.  ...  In [7, 13, 32, 31, 18, 17] , lowrank approximation or tensor decomposition is adopted to speed-up convolutional layers.  ... 
doi:10.1109/cvpr.2016.521 dblp:conf/cvpr/WuLWHC16 fatcat:yooibokbcvhxlhqdmjw7swegoq

T-Net: Parametrizing Fully Convolutional Nets With a Single High-Order Tensor

Jean Kossaifi, Adrian Bulat, Georgios Tzimiropoulos, Maja Pantic
2019 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
In this paper, we propose fully parametrizing Convolutional Neural Networks (CNNs) with a single, low-rank tensor.  ...  We study the case of networks with rich structure, namely Fully Convolutional CNNs, which we propose to parametrize them with a single 8−dimensional tensor.  ...  The authors in Lebedev et al. (2014) propose such parametrization of individual convolutional layers using CP decomposition with the goal of speeding them up.  ... 
doi:10.1109/cvpr.2019.00801 dblp:conf/cvpr/KossaifiBTP19 fatcat:of5uqx5jp5ewrof5bx57cmhkyu

Quantized Convolutional Neural Networks for Mobile Devices [article]

Jiaxiang Wu, Cong Leng, Yuhang Wang, Qinghao Hu, Jian Cheng
2016 arXiv   pre-print
Recently, convolutional neural networks (CNN) have demonstrated impressive performance in various computer vision tasks.  ...  Extensive experiments on the ILSVRC-12 benchmark demonstrate 4~6x speed-up and 15~20x compression with merely one percentage loss of classification accuracy.  ...  In [7, 13, 32, 31, 18, 17] , lowrank approximation or tensor decomposition is adopted to speed-up convolutional layers.  ... 
arXiv:1512.06473v3 fatcat:7ioc6nsqqne73iuldfozqtqmbu
« Previous Showing results 1 — 15 out of 718 results