688 Hits in 3.9 sec

Design of a 2-Bit Neural Network Quantizer for Laplacian Source

Zoran Perić, Milan Savić, Nikola Simić, Bojan Denić, Vladimir Despotović
2021 Entropy  
Here, we analyze in detail and design a 2-bit uniform quantization model for Laplacian source due to its significance in terms of implementation simplicity, which further leads to a shorter processing  ...  Moving from a full-precision neural network model to a lower representation by applying quantization techniques is a popular approach to facilitate this issue.  ...  The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.  ... 
doi:10.3390/e23080933 fatcat:5sizvsrauvdejevlvitddh4daa

Quantization of Weights of Neural Networks with Negligible Decreasing of Prediction Accuracy

Zoran Peric, Bojan Denic, Milan Savic, Milan Dincic, Darko Mihajlov
2021 Information Technology and Control  
We present a design approach for the memoryless Laplacian source with zero-mean and unit variance, which is based on iterative rule and uses the minimal mean-squared error distortion as a performance criterion  ...  Quantization and compression of neural network parameters using the uniform scalar quantization is carried out in this paper.  ...  Acknowledgement This work has been supported by the Ministry of Education, Science and Technological Development of the Republic of Serbia and by the Science Fund of the Republic of Serbia (Grant No. 6527104  ... 
doi:10.5755/j01.itc.50.3.28468 fatcat:kiemvc7llna6rjtuvx2ri6amwq

Symmetric Quantile Quantizer Parameterization for the Laplacian Source: Qualification for Contemporary Quantization Solutions

Zoran Perić, Jelena Nikolić, Danijela Aleksić, Anastasija Perić, A. M. Bastos Pereira
2021 Mathematical Problems in Engineering  
We suggest a simple method for offline precalculation of its parameters and we examine the inevitable loss of information introduced by SQQ, as an important part of bit optimization task at the traditional  ...  As a result, our SQQ outperforms SCQ in terms of signal-to-quantization noise ratio (SQNR).  ...  Acknowledgments is research was supported by the Science Fund of the Republic of Serbia, 6527104, AI-Com-in-AI.  ... 
doi:10.1155/2021/6647135 fatcat:2nqzx3c62fcwzh4ziuvfk75pxq

Design and Analysis of Binary Scalar Quantizer of Laplacian Source with Applications

Zoran Peric, Bojan Denic, Milan Savic, Vladimir Despotovic
2020 Information  
A compression method based on non-uniform binary scalar quantization, designed for the memoryless Laplacian source with zero-mean and unit variance, is analyzed in this paper.  ...  The motivation behind the binary quantization of neural network weights is the model compression by a factor of 32, which is crucial for implementation in mobile or embedded devices with limited memory  ...  Furthermore, we investigate the effect of clipping with the aim of reducing the quantization noise. Quantizers are designed for the memoryless Laplacian source with zero-mean and unit variance.  ... 
doi:10.3390/info11110501 fatcat:evmv3a243fd2rdp7f6miuhkpmm

Binary Quantization Analysis of Neural Networks Weights on MNIST Dataset

Zoran H. Peric, Bojan D. Denic, Milan S. Savic, Nikola J. Vucic, Nikola B. Simic
2021 Elektronika ir Elektrotechnika  
This paper considers the design of a binary scalar quantizer of Laplacian source and its application in compressed neural networks.  ...  Binary quantizers are further implemented for compressing neural network weights and its performance is analysed for a simple classification task.  ...  DESIGN OF BINARY QUANTIZER FOR THE REFERENCE VARIANCE Let us consider a symmetrical binary (N = 2 levels) scalar Binary Quantization Analysis of Neural Networks Weights on MNIST Dataset quantizer presented  ... 
doi:10.5755/j02.eie.28881 fatcat:bl77womljnh3ph6u3v6ihp2szm

Performance of Post-Training Two-Bits Uniform and Layer-Wise Uniform Quantization for MNIST Dataset from the Perspective of Support Region Choice

Stefan Tomić, Jelena Nikolić, Zoran Perić, Danijela Aleksić, Hao Gao
2022 Mathematical Problems in Engineering  
We provide experimental and theoretical results for a few significant cases of two-bits uniform quantizer design, where we assume that Laplacian source models the distribution of weights in our fully connected  ...  and the accuracy of the compressed neural network (NN) model.  ...  Acknowledgments is research was supported by the Science Fund of the Republic of Serbia, 6527104, AI-Com-in-AI.  ... 
doi:10.1155/2022/1463094 fatcat:qqbino4b3narhg6ai3swwdgmla

Iterative Algorithm for Parameterization of Two-Region Piecewise Uniform Quantizer for the Laplacian Source

Jelena Nikolić, Danijela Aleksić, Zoran Perić, Milan Dinčić
2021 Mathematics  
We believe that the resulting formulas for PWUQ design and performance assessment are greatly beneficial in neural networks where weights and activations are typically modelled by the Laplacian distribution  ...  For medium and high bit-rates, we demonstrated the convenience of our PWUQ over the uniform quantizer, paying special attention to the case where 99.99% of the signal amplitudes belong to the support region  ...  Acknowledgments: The authors would like to thank anonymous reviewers for their constructive comments. Conflicts of Interest: The authors declare no conflict of interest.  ... 
doi:10.3390/math9233091 fatcat:kxtd4ekgpndutdeuy4qbuxmfsi

End-to-end Optimized Video Compression with MV-Residual Prediction [article]

XiangJi Wu, Ziwen Zhang, Jie Feng, Lei Zhou, Junmin Wu
2020 arXiv   pre-print
A joint motion vector (MV) and residual prediction network MV-Residual is designed to extract the ensembled features of motion representations and residual information by treating the two successive frames  ...  The prior probability of the latent representations is modeled by a hyperprior autoencoder and trained jointly with the MV-Residual network.  ...  Introduction Recently, artificial neural networks (ANNs) have been applied to solve the image and video compression problem and a number of works have been proposed [11, 6, 2, 3, 5, 9, 18] .  ... 
arXiv:2005.12945v1 fatcat:xcibnnubn5dqnm42j2juot7iri

Pyramid based Progressive Transmission System with Hybrid Compression Scheme for Medical Images

H. K. Ravikiran, J. Jayanth
2021 SN Computer Science  
Telemedicine is an enabler for rural health care services where network quality varies over time and there is a need for robustness in presence of poor network quality and efficient use of bandwidth.  ...  In this work, a progressive transmission system with an efficient compression scheme is proposed for the effectual transmission of medical images over the internet with consideration for optimal use of  ...  for Women, Mysuru.  ... 
doi:10.1007/s42979-021-00534-7 fatcat:2uyrdffjdjd7beadfk53f6axlu

Forward Adaptive Dual-Mode Quantizer Based on the First-Degree Spline Approximation and Embedded G.711 Codec

Z. Peric, J. Nikolic, B. Denic, V. Despotovic
2019 Radioengineering  
, which is optimized for the assumed Laplacian source so that to provide a minimal mean-squared error distortion.  ...  The theoretical analysis in a wide dynamic range of input signal variances reveals that the proposed model of quantizer is superior versus the unrestricted G.711 quantizer as well as other similar baselines  ...  Acknowledgments This work was supported in part by the Ministry of Education and Science of the Republic of Serbia, grant no. TR32035 and TR32051 within the Technological Development Program.  ... 
doi:10.13164/re.2019.0729 fatcat:xo64ayqlnfaihgjk3qubku6p5a

Learned Image Compression with Discretized Gaussian-Laplacian-Logistic Mixture Model and Concatenated Residual Modules [article]

Haisheng Fu and Feng Liang and Jianping Lin and Bing Li and Mohammad Akbari and Jie Liang and Guohe Zhang and Dong Liu and Chengjie Tu and Jingning Han
2021 arXiv   pre-print
Besides, in the encoding/decoding network design part, we propose a concatenated residual blocks (CRB), where multiple residual blocks are serially connected with additional shortcut connections.  ...  coding (4:4:4 and 4:2:0) in terms of the PSNR and MS-SSIM.  ...  A powerful building block of many cuttingedge neural networks is the residual block first proposed in the ResNet [5] , which uses shortcut connections to facilitate the design and training of deep networks  ... 
arXiv:2107.06463v2 fatcat:wuz4h7tr2rc5nk7fp74gscy2ay

Reduced-Complexity End-to-End Variational Autoencoder for on Board Satellite Image Compression

Vinicius Alves de Oliveira, Marie Chabert, Thomas Oberlin, Charly Poulliat, Mickael Bruno, Christophe Latry, Mikael Carlavan, Simon Henrot, Frederic Falzon, Roberto Camarero
2021 Remote Sensing  
Recently, convolutional neural networks have been successfully applied to lossy image compression.  ...  Indeed, a statistical analysis performed on satellite images shows that the Laplacian distribution fits most features of their representation.  ...  The entropy measure, which quantifies the information contained in a source, provides a theoretical boundary for lossless compression, e.g., the lowest attainable compression bit-rate.  ... 
doi:10.3390/rs13030447 fatcat:jw252bqzxrfvtfhqnwqnwitp34

Lossless Compression of Mosaic Images with Convolutional Neural Network Prediction [article]

Seyed Mehdi Ayyoubzadeh, Xiaolin Wu
2020 arXiv   pre-print
We present a CNN-based predictive lossless compression scheme for raw color mosaic images of digital cameras.  ...  The key innovation of this paper is a high-order nonlinear CNN predictor of spatial-spectral mosaic patterns.  ...  They used two RNNs as encoder and decoder, a binarizer and a neural network for entropy coding.  ... 
arXiv:2001.10484v1 fatcat:ywdzv2nlyrepjctg6w6kcrffwa

Improved Deep Distributed Light Field Coding

M. Umair Mukati, Milan Stepanov, Giuseppe Valenzise, Soren Forchhammer, Frederic Dufaux
2021 IEEE Open Journal of Circuits and Systems  
Specifically, we train two deep neural networks to improve the two most critical parts of a distributed source coding scheme: the prediction of side information and the estimation of the uncertainty in  ...  These scenarios typically include low-power camera systems, for example, in wireless networks or multi-view video entertainment [2].  ...  Since the bitplanes for each frequency band are encoded one at a time by the LDPCA encoder, this results in a source code of length 12784 bits.  ... 
doi:10.1109/ojcas.2021.3073252 fatcat:ul7f7nvthvgubk3kz6lmweferu

Whether the Support Region of Three-Bit Uniform Quantizer Has a Strong Impact on Post-Training Quantization for MNIST Dataset?

Jelena Nikolić, Zoran Perić, Danijela Aleksić, Stefan Tomić, Aleksandra Jovanović
2021 Entropy  
Driven by the need for the compression of weights in neural networks (NNs), which is especially beneficial for edge devices with a constrained resource, and by the need to utilize the simplest possible  ...  threshold value of the quantizer to achieve some predefined accuracy of the quantized neural network (QNN).  ...  The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.  ... 
doi:10.3390/e23121699 pmid:34946005 pmcid:PMC8700806 fatcat:y5kgtozc2zeblluufhxlutlg6q
« Previous Showing results 1 — 15 out of 688 results