A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Compressing Deep Networks by Neuron Agglomerative Clustering
2020
Sensors
Preserved Fulltext
In this paper, to tackle this problem, we introduce a method for compressing the structure and parameters of DNNs based on neuron agglomerative clustering (NAC). ...
Specifically, we utilize the agglomerative clustering algorithm to find similar neurons, while these similar neurons and the connections linked to them are then agglomerated together. ...
Acknowledgments: The authors would like to thank the guest editors and the anonymous reviewers for their work and time on the publication of this paper. ...
doi:10.3390/s20216033
pmid:33114078
pmcid:PMC7660330
fatcat:hetsfskkifdlbhdgkbdh66ked4
Multi-Task Zipping via Layer-wise Neuron Sharing
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We propose Multi-Task Zipping (MTZ), a framework to automatically merge correlated, pre-trained deep neural networks for cross-model compression. ...
two networks with <0.5% increase in the test errors for both tasks. ...
Conclusion We propose MTZ, a framework to automatically merge multiple correlated, well-trained deep neural networks for cross-model compression via neuron sharing. ...
arXiv:1805.09791v2
fatcat:mtsfbjfuqzawlixggzczyw22tq
Diversity Networks: Neural Network Compression Using Determinantal Point Processes
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We introduce Divnet, a flexible technique for learning networks with diverse neurons. Divnet models neuronal diversity by placing a Determinantal Point Process (DPP) over neurons in a given layer. ...
We present experimental results to corroborate our claims: for pruning neural networks, Divnet is seen to be notably superior to competing approaches. ...
These methods focus on deleting parameters whose removal influences the network the least, while Divnet seeks diversity and merges similar neurons; these methods can thus be used in conjunction with ours ...
arXiv:1511.05077v6
fatcat:l4laul3l7vfdleqthpqq3fmh64
Literature Review of Deep Network Compression
2021
Informatics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
In this paper, we present an overview of popular methods and review recent works on compressing and accelerating deep neural networks. ...
Deep networks often possess a vast number of parameters, and their significant redundancy in parameterization has become a widely-recognized property. ...
[39] introduced Divnet, which selects a subset of diverse neurons and merges similar neurons into one. ...
doi:10.3390/informatics8040077
fatcat:u2dzzibapnf2dbjdqkvgl3pztu
Neural Behavior-Based Approach for Neural Network Pruning
2020
IEICE transactions on information and systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
neurons having the similar behavioral vectors. ...
Therefore, the proposed method can reduce the number of neurons with small sacrifice of accuracy without retraining. Our method can be applied for compressing convolutional layers as well. ...
Introduction Deep neural networks (DNNs) have been showing dominant performances in the machine learning tasks. The key is the scale of the models. ...
doi:10.1587/transinf.2019edp7177
fatcat:uawh24gzobhenhspsezyrg3qxm
RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Pruning Deep Neural Networks (DNNs) is a prominent field of study in the goal of inference runtime acceleration. In this paper, we introduce a novel data-free pruning protocol RED++. ...
Only requiring a trained neural network, and not specific to DNN architecture, we exploit an adaptive data-free scalar hashing which exhibits redundancies among neuron weight values. ...
The merged network weightsW are obtained by merging the first 2 neurons of layer l and updating the consecutive layer by simply summing the corresponding weights. ...
arXiv:2110.01397v1
fatcat:qhy4hn6bifhdjg53v43ztstqx4
Incremental Layers Resection: A Novel Method to Compress Neural Networks
2019
IEEE Access
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
In recent years, deep neural networks (DNNs) have been widely applied in many areas, such as computer vision and pattern recognition. ...
Extensive experiments demonstrate that, compared to the original networks, the compressed ones by ILR need only about half of the storage space and have higher inference speed. ...
In [19] , a similar neurons merging algorithm has been proposed to merge the neurons with similar performance in the same layer. ...
doi:10.1109/access.2019.2952615
fatcat:ipm6yjvhkzfvzcmmyavm2wxgoq
DeepAbstract: Neural Network Abstraction for Accelerating Verification
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
While abstraction is a classic tool of verification to scale it up, it is not used very often for verifying neural networks. ...
We introduce an abstraction framework applicable to fully-connected feed-forward neural networks based on clustering of neurons that behave similarly on some inputs. ...
Neurons 1-7 and 9 are unaffected by the merging procedure. For neuron 11, we havel 11 = 5 andũ 11 = 13, and for neuron 12,l 12 = 0 andũ 12 = 4. ...
arXiv:2006.13735v1
fatcat:piex3f2zmjgrncwiflqrxx5fxa
Deep Autoencoder-Based Image Compression using Multi-Layer Perceptrons
2020
International journal of soft computing and engineering
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In this research, a deep autoencoder-based multi-layer feed-forward neural network has been proposed to achieve image compression. ...
The Artificial Neural Network is one of the heavily used alternatives for solving complex problems in machine learning and deep learning. ...
In this research, we used a deep neural network that consists of multiple hidden layers. When it comes to image compression, deep learning is heavily used. ...
doi:10.35940/ijsce.e3357.039620
fatcat:o3qzl5ufmzhobg5h7gqofedxpe
Deep Learning in Biological Data Analysis
2017
MOJ Proteomics & Bioinformatics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
Then the downstream layers of deep neural network can learn from such similarity information and draw patterns from the transformed datasets. ...
Feature learning With its deep structures, the low (beginning or closer to input) neuron layers of a deep neural network may not directly perform a supervised learning task. ...
doi:10.15406/mojpb.2017.05.00148
fatcat:gauid6m36vfhjociw3fbtdw2g4
Go Wide, Then Narrow: Efficient Training of Deep Thin Networks
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this paper, we propose an efficient method to train a deep thin network with a theoretic guarantee. Our method is motivated by model compression. It consists of three stages. ...
For deploying a deep learning model into production, it needs to be both accurate and compact to meet the latency and memory constraints. ...
In this paper, we propose a generic algorithm to train deep thin networks with a theoretical guarantee. Our method is motivated by model compression. ...
arXiv:2007.00811v2
fatcat:zz4tc7tmi5e6rmvd66hulk3ri4
Compressing Deep Neural Networks via Layer Fusion
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
From experiments on CIFAR-10, we find that various deep convolution neural networks can remain within 2% accuracy points of the original networks up to a compression ratio of 3.33 when iteratively retrained ...
For experiments on the WikiText-2 language modelling dataset where pretrained transformer models are used, we achieve compression that leads to a network that is 20% of its original size while being within ...
We find that merging the most similar layers during the retraining process of already deep pretrained neural network leads to competitive performance when compared against the original network, while maintaining ...
arXiv:2007.14917v1
fatcat:7j3gh3gudbgm7escljljdrzjme
Complexity Reduction of Learned In-Loop Filtering in Video Coding
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2022 pre-print arXiv:2203.08650v1 fatcat:tadwimyszrftzapiynguotakmu
Through initial tests we find that network parameters can be significantly reduced with a minimal impact on network performance. ...
This is done through a three-step training process of magnitude-guidedweight pruning, insignificant neuron identification and removal, and fine-tuning. ...
ACKNOWLEDGEMENTS This work was supported by the UK Engineering and Physical Sciences Research Council (EPSRC) grant 2246465 for Queen Mary University of London, Multimedia and Vision Group (MMV), and the ...
arXiv:2203.08650v2
fatcat:2llbwxyjl5fpvfmor67xe4ixcq
Deep Epitome for Unravelling Generalized Hamming Network: A Fuzzy Logic Interpretation of Deep Learning
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
For network visualization, the constructed deep epitomes at each layer provide a visualization of network internal representation that does not rely on the input data. ...
Moreover, deep epitomes allows the direct extraction of features in just one step, without resorting to regularized optimizations used in existing visualization tools. ...
Then the negative GHD is used to quantify the similarity between neuron inputs x and weights w: −g(w, x) = 2 L w · x − 1 L L l=1 w l − 1 L L l=1 x l , (1) in which L denotes the length of neuron weights ...
arXiv:1711.05397v1
fatcat:gdqvsja4xjdibdhimgmkgtvfc4
Lossless Compression of Structured Convolutional Models via Lifting
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2020 pre-print arXiv:2007.06567v1 fatcat:bhwsaat56nhyte6veptwt7c2z4
We demonstrate through experiments that such compression can lead to significant speedups of structured convolutional models, such as various Graph Neural Networks, across various tasks, such as molecule ...
Inspired by lifting, we introduce a simple and efficient technique to detect the symmetries and compress the neural models without loss of any information. ...
Algorithm 1 Neural network compression based on detecting functional symmetries 1: function COMPRESS (Network)
2:
N ← neurons (Network)
3:
N ← topologicOrder (N )
4:
W ← weights (Network)
5:
M ...
arXiv:2007.06567v2
fatcat:3ea3dzdeufd4bjnn5bsjwf5cgi
« Previous
Showing results 1 — 15 out of 5,420 results