A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Filters
Training with Quantization Noise for Extreme Model Compression
[article]
2021
arXiv
pre-print
Preserved Fulltext
Other Versions
Controlling the amount of noise and its form allows for extreme compression rates while maintaining the performance of the original model. ...
A standard solution is to train networks with Quantization Aware Training, where the weights are quantized during training and the gradients approximated with the Straight-Through Estimator. ...
FINETUNING WITH QUANT-NOISE FOR POST-PROCESSING QUANTIZATION We explore taking existing models and post-processing with Quant-Noise instead of training from scratch. ...
arXiv:2004.07320v3
fatcat:lexa56jkmfffvgnyl3f7dfonkq
Extremely Low Footprint End-to-End ASR System for Smart Device
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We design cross-layer weight sharing to improve parameter efficiency and further exploit model compression methods including sparsification and quantization, to reduce memory storage and boost decoding ...
We propose an extremely low footprint E2E ASR system for smart devices, to achieve the goal of satisfying resource constraints without sacrificing recognition accuracy. ...
When combining weight sharing with model compression, we obtain an extremely low footprint model. ...
arXiv:2104.05784v5
fatcat:ajd2eiv22fbelhlrouxlaybycu
Soft then Hard: Rethinking the Quantization in Neural Image Compression
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Training with additive uniform noise approximates the quantization error variationally but suffers from the train-test mismatch. ...
We thus propose a novel soft-then-hard quantization strategy for neural image compression that first learns an expressive latent space softly, then closes the train-test mismatch with hard quantization ...
Other Quantization Methods As introduced in Section 2.1, training the lossy image compression model with additive uniform noise approximates the quantization error variationally. ...
arXiv:2104.05168v3
fatcat:wzr4mtpzyfggpa52z3vtre3c2a
Fidelity-Controllable Extreme Image Compression with Generative Adversarial Networks
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We propose a GAN-based image compression method working at extremely low bitrates below 0.1bpp. Most existing learned image compression methods suffer from blur at extremely low bitrates. ...
First, GAN makes training unstable. Second, the reconstructions often contain unpleasing noise or artifacts. ...
For the first training, λ (1) d depends on our ideal bitrates. We trained four models with different compression rates. ...
arXiv:2008.10314v1
fatcat:r66e2peqcfgapejekayrk7sclq
Low Bit-Rate Compression Image Restoration through Subspace Joint Regression Learning
2018
IEICE transactions on information and systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Experimental results demonstrate the proposed method achieves favorable performance compared with other leading methods. key words: compression image restoration, subspace regression learning, non-local ...
means denoising, extreme support vector regression ...
Quantization noise sometimes need be modeled, and it can be used as constraint condition to remove compression artifacts [1] . ...
doi:10.1587/transinf.2017edl8278
fatcat:24q4472merfinnm6itipyyerr4
Iterative Low-Rank Approximation for CNN Compression
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Since classification and object detection are the most favored tasks for embedded devices, we demonstrate the effectiveness of our approach by compressing AlexNet, VGG-16, YOLOv2 and Tiny YOLO networks ...
We demonstrate higher compression ratio providing less accuracy loss. ...
The results show the superiority of our iterative approach with gradual compression in comparison with non-repetitive ones. ...
arXiv:1803.08995v2
fatcat:lstgp7l4gngbhgrfg547pec4qu
Quantization in Layer's Input is Matter
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
In this paper, we will show that the quantization in layer's input is more important than parameters' quantization for loss function. ...
And the algorithm which is based on the layer's input quantization error is better than hessian-based mixed precision layout algorithm. ...
A well-trained model corresponds to an extreme point of the loss function. ...
arXiv:2202.05137v1
fatcat:mpeb2e55cjeb5kewt6rsxegwqy
QUANOS- Adversarial Noise Sensitivity Driven Hybrid Quantization of Neural Networks
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
A key advantage of QUANOS is that it does not rely on a pre-trained model and can be applied in the initial stages of training. ...
We identify a novel noise stability metric (ANS) for DNNs, i.e., the sensitivity of each layer's computation to adversarial noise. ...
The extreme case is that of a 1-bit quantized DNN. Such binary quantized models are usually trained from scratch to obtain competitive accuracy as that of a full-precision model [8] . ...
arXiv:2004.11233v2
fatcat:dpaio4flvrds3m6ga6ghhog54e
Combined compression and denoising of images using vector quantization
1998
Applications of Digital Image Processing XXI
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2012; you can also visit the original URL.
The file type is application/pdf.
The idea is to train a VQ on pairs of noisy and clean images. When presented with a noisy image, our VQ-based system estimates the noise variance and then performs joint denoising and compression. ...
optimal quantizer for the estimate. ...
Combined compression and denoising The signal-to-noise ratio (SNR), measured always with respect to the clean image, is defined as SNR = 10 log10 signal power dB. noise power Training blocks of size 2 ...
doi:10.1117/12.323206
fatcat:fsjfdiqejbfy7ofd3bemd3oaxq
Countering Adversarial Examples: Combining Input Transformation and Noisy Training
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
We fine-tune the pre-trained model by training with images encoded at different compression levels, thus generating multiple classifiers. ...
Considering compression as a data augmentation strategy, we then combine our model-agnostic preprocess with noisy training. ...
The model set is realized by fine-tuning the models with compressed images at different compression level. The initial model used for fine-tuning is the pretrained model on benign images. ...
arXiv:2106.13394v1
fatcat:4bp3e7zf4zaz7m5m425gx25sby
Wideband and Entropy-Aware Deep Soft Bit Quantization
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Our method is trained end-to-end with quantization- and entropy-aware augmentations to the loss function and is used at inference in conjunction with source coding to achieve near-optimal compression gains ...
To efficiently train our method, we prove and verify that a fixed feature space quantization scheme is sufficient for efficient learning. ...
The data used to train all models comes from transmissions across i.i.d. Rayleigh fading channels, where h i ∼ N C (0, 1) and the noise n i ∼ N C (0, σ n ). ...
arXiv:2110.09541v1
fatcat:2tjda3viznbvrlmvpwpyttqazm
Evaluation of speech quality using digital watermarking
2004
IEICE Electronics Express
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
The existing objective quality assessment methods require either the original speech or complicated computation model, which makes some applications of quality evaluation impossible. ...
Our method does not need original signal or computation model. The experimental results show that the method yields accurate quality scores which are very close to the results of PESQ. ...
For the distortions, we set SNR from 5 to 50 with an interval of 5 for Gaussian noise; bit rate from 32 to 320 Kbps with an interval of 32 Kbps for MP3 compression; and threshold frequency from 1 to 29 ...
doi:10.1587/elex.1.380
fatcat:v6fch73gmzbgtpi57p76tt6kdm
SEOFP-NET: Compression and Acceleration of Deep Neural Networks for Speech Enhancement Using Sign-Exponent-Only Floating-Points
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
The proposed method compressed the sizes of deep neural network (DNN)-based speech enhancement models by quantizing the fraction bits of single-precision floating-point parameters during training. ...
The results also verify that the proposed SEOFP-NET can cooperate with other efficiency strategies to achieve a synergy effect for model compression. ...
The model then used the updated parameter with value 0.125000 for the next iteration. The algorithm keeps quantizing the model until the end of the training. ...
arXiv:2111.04436v1
fatcat:53l4nk4okffgjje3lavuoa2c3y
Volumetric Medical Images Lossy Compression using Stationary Wavelet Transform and Linde-Buzo-Gray Vector Quantization
2017
Research Journal of Applied Sciences Engineering and Technology
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
The system makes use of a combination of Linde-Buzo-Gray vector quantization technique for lossy compression along with Arithmetic coding and Huffman coding for lossless compression. ...
This study proposes a novel system for efficient lossy volumetric medical image compression using Stationary Wavelet Transform and Linde-Buzo-Gray for Vector Quantization. ...
Where the results in Table 1 show the compression ratio, the peak signal-to-noise ratio and the running time for the proposed system using SWT with a combination of the LBG Vector Quantization and the ...
doi:10.19026/rjaset.14.5134
fatcat:kwbmjrxpfffrbk3shrm5q7ti34
Deep Joint Source-Channel Coding for Wireless Image Retrieval
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We first note that reconstructing the original image is not needed for retrieval tasks; hence, we introduce a deep neutral network (DNN) based compression scheme targeting the retrieval task. ...
Motivated by surveillance applications with wireless cameras or drones, we consider the problem of image retrieval over a wireless channel. ...
For quantization, we adopt the quantization noise from [11] as a differentiable approximation of this operation during training. ...
arXiv:1910.12703v1
fatcat:3bm2ndovkfhepnsc7dvmhox77e
« Previous
Showing results 1 — 15 out of 7,500 results