7,500 Hits in 3.3 sec

Training with Quantization Noise for Extreme Model Compression [article]

Angela Fan, Pierre Stock, Benjamin Graham, Edouard Grave, Remi Gribonval, Herve Jegou, Armand Joulin
2021 arXiv   pre-print
Controlling the amount of noise and its form allows for extreme compression rates while maintaining the performance of the original model.  ...  A standard solution is to train networks with Quantization Aware Training, where the weights are quantized during training and the gradients approximated with the Straight-Through Estimator.  ...  FINETUNING WITH QUANT-NOISE FOR POST-PROCESSING QUANTIZATION We explore taking existing models and post-processing with Quant-Noise instead of training from scratch.  ... 
arXiv:2004.07320v3 fatcat:lexa56jkmfffvgnyl3f7dfonkq

Extremely Low Footprint End-to-End ASR System for Smart Device [article]

Zhifu Gao, Yiwu Yao, Shiliang Zhang, Jun Yang, Ming Lei, Ian McLoughlin
2021 arXiv   pre-print
We design cross-layer weight sharing to improve parameter efficiency and further exploit model compression methods including sparsification and quantization, to reduce memory storage and boost decoding  ...  We propose an extremely low footprint E2E ASR system for smart devices, to achieve the goal of satisfying resource constraints without sacrificing recognition accuracy.  ...  When combining weight sharing with model compression, we obtain an extremely low footprint model.  ... 
arXiv:2104.05784v5 fatcat:ajd2eiv22fbelhlrouxlaybycu

Soft then Hard: Rethinking the Quantization in Neural Image Compression [article]

Zongyu Guo, Zhizheng Zhang, Runsen Feng, Zhibo Chen
2021 arXiv   pre-print
Training with additive uniform noise approximates the quantization error variationally but suffers from the train-test mismatch.  ...  We thus propose a novel soft-then-hard quantization strategy for neural image compression that first learns an expressive latent space softly, then closes the train-test mismatch with hard quantization  ...  Other Quantization Methods As introduced in Section 2.1, training the lossy image compression model with additive uniform noise approximates the quantization error variationally.  ... 
arXiv:2104.05168v3 fatcat:wzr4mtpzyfggpa52z3vtre3c2a

Fidelity-Controllable Extreme Image Compression with Generative Adversarial Networks [article]

Shoma Iwai, Tomo Miyazaki, Yoshihiro Sugaya, Shinichiro Omachi
2020 arXiv   pre-print
We propose a GAN-based image compression method working at extremely low bitrates below 0.1bpp. Most existing learned image compression methods suffer from blur at extremely low bitrates.  ...  First, GAN makes training unstable. Second, the reconstructions often contain unpleasing noise or artifacts.  ...  For the first training, λ (1) d depends on our ideal bitrates. We trained four models with different compression rates.  ... 
arXiv:2008.10314v1 fatcat:r66e2peqcfgapejekayrk7sclq

Low Bit-Rate Compression Image Restoration through Subspace Joint Regression Learning

Zongliang GAN
2018 IEICE transactions on information and systems  
Experimental results demonstrate the proposed method achieves favorable performance compared with other leading methods. key words: compression image restoration, subspace regression learning, non-local  ...  means denoising, extreme support vector regression  ...  Quantization noise sometimes need be modeled, and it can be used as constraint condition to remove compression artifacts [1] .  ... 
doi:10.1587/transinf.2017edl8278 fatcat:24q4472merfinnm6itipyyerr4

Iterative Low-Rank Approximation for CNN Compression [article]

Maksym Kholiavchenko
2019 arXiv   pre-print
Since classification and object detection are the most favored tasks for embedded devices, we demonstrate the effectiveness of our approach by compressing AlexNet, VGG-16, YOLOv2 and Tiny YOLO networks  ...  We demonstrate higher compression ratio providing less accuracy loss.  ...  The results show the superiority of our iterative approach with gradual compression in comparison with non-repetitive ones.  ... 
arXiv:1803.08995v2 fatcat:lstgp7l4gngbhgrfg547pec4qu

Quantization in Layer's Input is Matter [article]

Daning Cheng, WenGuang Chen
2022 arXiv   pre-print
In this paper, we will show that the quantization in layer's input is more important than parameters' quantization for loss function.  ...  And the algorithm which is based on the layer's input quantization error is better than hessian-based mixed precision layout algorithm.  ...  A well-trained model corresponds to an extreme point of the loss function.  ... 
arXiv:2202.05137v1 fatcat:mpeb2e55cjeb5kewt6rsxegwqy

QUANOS- Adversarial Noise Sensitivity Driven Hybrid Quantization of Neural Networks [article]

Priyadarshini Panda
2020 arXiv   pre-print
A key advantage of QUANOS is that it does not rely on a pre-trained model and can be applied in the initial stages of training.  ...  We identify a novel noise stability metric (ANS) for DNNs, i.e., the sensitivity of each layer's computation to adversarial noise.  ...  The extreme case is that of a 1-bit quantized DNN. Such binary quantized models are usually trained from scratch to obtain competitive accuracy as that of a full-precision model [8] .  ... 
arXiv:2004.11233v2 fatcat:dpaio4flvrds3m6ga6ghhog54e

Combined compression and denoising of images using vector quantization

Kannan Panchapakesan, Ali Bilgin, David G. Sheppard, Michael W. Marcellin, Bobby R. Hunt, Andrew G. Tescher
1998 Applications of Digital Image Processing XXI  
The idea is to train a VQ on pairs of noisy and clean images. When presented with a noisy image, our VQ-based system estimates the noise variance and then performs joint denoising and compression.  ...  optimal quantizer for the estimate.  ...  Combined compression and denoising The signal-to-noise ratio (SNR), measured always with respect to the clean image, is defined as SNR = 10 log10 signal power dB. noise power Training blocks of size 2  ... 
doi:10.1117/12.323206 fatcat:fsjfdiqejbfy7ofd3bemd3oaxq

Countering Adversarial Examples: Combining Input Transformation and Noisy Training [article]

Cheng Zhang, Pan Gao
2021 arXiv   pre-print
We fine-tune the pre-trained model by training with images encoded at different compression levels, thus generating multiple classifiers.  ...  Considering compression as a data augmentation strategy, we then combine our model-agnostic preprocess with noisy training.  ...  The model set is realized by fine-tuning the models with compressed images at different compression level. The initial model used for fine-tuning is the pretrained model on benign images.  ... 
arXiv:2106.13394v1 fatcat:4bp3e7zf4zaz7m5m425gx25sby

Wideband and Entropy-Aware Deep Soft Bit Quantization [article]

Marius Arvinte, Jonathan I. Tamir
2021 arXiv   pre-print
Our method is trained end-to-end with quantization- and entropy-aware augmentations to the loss function and is used at inference in conjunction with source coding to achieve near-optimal compression gains  ...  To efficiently train our method, we prove and verify that a fixed feature space quantization scheme is sufficient for efficient learning.  ...  The data used to train all models comes from transmissions across i.i.d. Rayleigh fading channels, where h i ∼ N C (0, 1) and the noise n i ∼ N C (0, σ n ).  ... 
arXiv:2110.09541v1 fatcat:2tjda3viznbvrlmvpwpyttqazm

Evaluation of speech quality using digital watermarking

Libin Cai, Jiying Zhao
2004 IEICE Electronics Express  
The existing objective quality assessment methods require either the original speech or complicated computation model, which makes some applications of quality evaluation impossible.  ...  Our method does not need original signal or computation model. The experimental results show that the method yields accurate quality scores which are very close to the results of PESQ.  ...  For the distortions, we set SNR from 5 to 50 with an interval of 5 for Gaussian noise; bit rate from 32 to 320 Kbps with an interval of 32 Kbps for MP3 compression; and threshold frequency from 1 to 29  ... 
doi:10.1587/elex.1.380 fatcat:v6fch73gmzbgtpi57p76tt6kdm

SEOFP-NET: Compression and Acceleration of Deep Neural Networks for Speech Enhancement Using Sign-Exponent-Only Floating-Points [article]

Yu-Chen Lin, Cheng Yu, Yi-Te Hsu, Szu-Wei Fu, Yu Tsao, Tei-Wei Kuo
2021 arXiv   pre-print
The proposed method compressed the sizes of deep neural network (DNN)-based speech enhancement models by quantizing the fraction bits of single-precision floating-point parameters during training.  ...  The results also verify that the proposed SEOFP-NET can cooperate with other efficiency strategies to achieve a synergy effect for model compression.  ...  The model then used the updated parameter with value 0.125000 for the next iteration. The algorithm keeps quantizing the model until the end of the training.  ... 
arXiv:2111.04436v1 fatcat:53l4nk4okffgjje3lavuoa2c3y

Volumetric Medical Images Lossy Compression using Stationary Wavelet Transform and Linde-Buzo-Gray Vector Quantization

Hend A. Elsayed, Omar G. Abood, Shawkat K. Guirguis
2017 Research Journal of Applied Sciences Engineering and Technology  
The system makes use of a combination of Linde-Buzo-Gray vector quantization technique for lossy compression along with Arithmetic coding and Huffman coding for lossless compression.  ...  This study proposes a novel system for efficient lossy volumetric medical image compression using Stationary Wavelet Transform and Linde-Buzo-Gray for Vector Quantization.  ...  Where the results in Table 1 show the compression ratio, the peak signal-to-noise ratio and the running time for the proposed system using SWT with a combination of the LBG Vector Quantization and the  ... 
doi:10.19026/rjaset.14.5134 fatcat:kwbmjrxpfffrbk3shrm5q7ti34

Deep Joint Source-Channel Coding for Wireless Image Retrieval [article]

Mikolaj Jankowski, Deniz Gunduz, Krystian Mikolajczyk
2019 arXiv   pre-print
We first note that reconstructing the original image is not needed for retrieval tasks; hence, we introduce a deep neutral network (DNN) based compression scheme targeting the retrieval task.  ...  Motivated by surveillance applications with wireless cameras or drones, we consider the problem of image retrieval over a wireless channel.  ...  For quantization, we adopt the quantization noise from [11] as a differentiable approximation of this operation during training.  ... 
arXiv:1910.12703v1 fatcat:3bm2ndovkfhepnsc7dvmhox77e
« Previous Showing results 1 — 15 out of 7,500 results