884 Hits in 5.7 sec

Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithms [article]

Mahdi Haghifam, Jeffrey Negrea, Ashish Khisti, Daniel M. Roy, Gintare Karolina Dziugaite
2020 arXiv   pre-print
We first show that these new bounds based on the conditional mutual information are tighter than those based on the unconditional mutual information.  ...  Zou (2016) and Xu and Raginsky (2017) provides bounds on the generalization error of a learning algorithm in terms of the mutual information between the algorithm's output and the training sample.  ...  Generalization bounds for noisy, iterative algorithms We apply this new class of generalization bounds to non-convex learning.  ... 
arXiv:2004.12983v2 fatcat:lz4qf6dx6feevi66bmeov52rmu

Generalization Bounds for Noisy Iterative Algorithms Using Properties of Additive Noise Channels [article]

Hao Wang, Rui Gao, Flavio P. Calmon
2021 arXiv   pre-print
We derive distribution-dependent generalization bounds by connecting noisy iterative algorithms to additive noise channels found in communication and information theory.  ...  In this paper, we analyze the generalization of models trained by noisy iterative algorithms.  ...  Calmon also acknowledges a gift from Google Faculty Research Award and an Amazon Research Award.  ... 
arXiv:2102.02976v3 fatcat:k3mbx7yzhzckvcea2xmnli5pb4

Divergence Measures: Mathematical Foundations and Applications in Information-Theoretic and Statistical Problems

Igal Sason
2022 Entropy  
Data science, information theory, probability theory, statistical learning, statistical signal processing, and other related disciplines greatly benefit from non-negative measures of dissimilarity between  ...  Acknowledgments: The Guest Editor is grateful to all the authors for their contributions to this Special Issue, to the anonymous peer-reviewers for their timely reports and constructive feedback.  ...  iterative algorithms for estimating a distribution from an incomplete data, and other sorts of problems.  ... 
doi:10.3390/e24050712 pmid:35626595 pmcid:PMC9141399 fatcat:6lgxxon75zd6lk6fkzbrip5nyy

Conditioning and Processing: Techniques to Improve Information-Theoretic Generalization Bounds

Hassan Hafez-Kolahi, Zeinab Golgooni, Shohreh Kasaei, Mahdieh Soleymani
2020 Neural Information Processing Systems  
This approach provides an insight into learning algorithms by considering the mutual information between the model and the training set.  ...  These techniques can be used to improve the bounds by either sharpening them or increasing their applicability.  ...  Acknowledgments and Disclosure of Funding There are no financial conflicts of interest to disclose.  ... 
dblp:conf/nips/Hafez-KolahiGKS20 fatcat:7ilao7v3gvb3piqpcpk55shwge

Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation [article]

Rushi Jiao, Yichi Zhang, Le Ding, Rong Cai, Jicong Zhang
2022 arXiv   pre-print
Recent success of deep learning-based segmentation methods usually relies on a large amount of labeled data, which is particularly difficult and costly to obtain especially in the medical imaging domain  ...  Semi-supervised learning has emerged as an appealing strategy and been widely applied to medical image segmentation tasks to train deep models with limited annotations.  ...  We also appreciate the efforts of literature collection and code implementations of SSL4MIS 6 and several public benchmarks.  ... 
arXiv:2207.14191v1 fatcat:k47z5cbqbvhp7lzzhhfxtpt2wa

Data-driven multichannel superresolution with application to video sequences

H. Shekarforoush, R. Chellappa
1999 Optical Society of America. Journal A: Optics, Image Science, and Vision  
Based on a generalization of Papoulis' sampling theorem, nonuniform samples of multiple channels are merged to generate high-resolution data.  ...  The method is therefore designed to perform under practical conditions of noise and other degradations.  ...  Therefore an advantage of our superresolution algorithm is its capability of resolving visual information in highly noisy data such as the PREDA-TOR video.  ... 
doi:10.1364/josaa.16.000481 fatcat:2nbdfhgg7radhhgu7x5qr7kjii

Investigations on exemplar-based features for speech recognition towards thousands of hours of unsupervised, noisy data

Georg Heigold, Patrick Nguyen, Mitchel Weintraub, Vincent Vanhoucke
2012 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)  
Exemplar-based approaches are an attractive alternative, in particular if massive data and computational power are available. Yet, most of the data at Google are unsupervised and noisy.  ...  A log-linear rescoring framework is used to combine the exemplar-based features on the word level with the first-pass model.  ...  The optimization is done with the general-purpose algorithms L-BFGS or Rprop.  ... 
doi:10.1109/icassp.2012.6288904 dblp:conf/icassp/HeigoldNWV12 fatcat:5kc3e7yqsnf63n4y3v43yzxo4i

RF Tomography in Free Space: Experimental Validation of the Forward Model and an Inversion Algorithm Based on the Algebraic Reconstruction Technique

V. Picco, T. Negishi, S. Nishikata, D. Spitzer, D. Erricolo
2013 International Journal of Antennas and Propagation  
Experimental data are used to validate a novel inversion scheme, based on the algebraic reconstruction technique.  ...  The proposed method is improved by introducing physical bounds on the solution returned.  ...  An algorithm based on the Algebraic Reconstruction Technique, augmented with the enforcement of a physical bound, satisfied our requirements.  ... 
doi:10.1155/2013/528347 fatcat:xqkjko362ve7rbuagu5hta2zha

Signal Recovery on Incoherent Manifolds [article]

Chinmay Hegde, Richard G. Baraniuk
2012 arXiv   pre-print
Suppose that we observe noisy linear measurements of an unknown signal that can be modeled as the sum of two component signals, each of which arises from a nonlinear sub-manifold of a high dimensional  ...  SPIN significantly extends the scope of current recovery models and algorithms for low dimensional linear inverse problems and matches (or exceeds) the current state of the art in terms of performance.  ...  SPIN is an iterative gradient projection algorithm and requires as input parameters the number of iterations T and the gradient step size η.  ... 
arXiv:1202.1595v2 fatcat:nvnafpjihnfv5csltpdm4htguu

Leaf Image Segmentation Based On the Combination of Wavelet Transform and K Means Clustering

N Valliammal, Dr.S.N.Geethalakshmi
2012 International Journal of Advanced Research in Artificial Intelligence (IJARAI)  
In recent years, more and more attention has been paid to combine segmentation algorithms and information from multiple feature spaces (e.g. color, texture, and pattern) in order to improve segmentation  ...  Although many methods are proposed, it is still difficult to accurately segment an arbitrary image by one particular method.  ...  Thus, mutual information is positive and bounded by {H(X),H(Y)}_log2(n). D.  ... 
doi:10.14569/ijarai.2012.010307 fatcat:hqncjinirfhixiwajfnohxqrgi

A Survey of Label-noise Representation Learning: Past, Present and Future [article]

Bo Han, Quanming Yao, Tongliang Liu, Gang Niu, Ivor W. Tsang, James T. Kwok, Masashi Sugiyama
2021 arXiv   pre-print
Based on the theoretical guidance, we categorize different LNRL methods into three directions.  ...  However, statistical-learning-based methods may not train deep learning models robustly with these noisy labels.  ...  Based on the above generative process, we can deduce the evidence lower bound (ELBO) [74] to approximate the log-likelihood of the noisy data. Fine-tuning Revision Xia et al.  ... 
arXiv:2011.04406v2 fatcat:76np6wyzvvag7ehy23cwyzdozm

Lp-Norm Inversion of Gravity Data Using Adaptive Differential Evolution

Tao Song, Xing Hu, Wei Du, Lianzheng Cheng, Tiaojie Xiao, Qian Li
2021 Applied Sciences  
term is implemented to sharpen the boundary of density distribution.  ...  In the synthetic anomaly case, both noise-free and noisy data sets are considered.  ...  Conclusions In this work, an attempt was made to test the applicability and effectiveness of a tive DE on Lp-norm gravity inversion.  ... 
doi:10.3390/app11146485 fatcat:jhhcq6abv5cbvnokqnhtvsdiwm

Technical Program

2021 2020 IEEE Information Theory Workshop (ITW)  
Our proof is based entirely on convex optimization and functional analysis, without resorting to any "quantum" arguments.  ...  It was also shown that the same signal is optimal in the sense of maximizing the mutual information between the identity of the state and the measurements.  ...  First, we recover the bounds based on the individual sample mutual information from Bu et al. [2] and on a random subset of the dataset from Negrea et al. [3].  ... 
doi:10.1109/itw46852.2021.9457668 fatcat:j425ygeajrbd5esztbe5zgygty

Blind restoration for nonuniform aerial images using nonlocal Retinex model and shearlet-based higher-order regularization

Rui Chen, Huizhu Jia, Xiaodong Xie, Wen Gao
2017 Journal of Electronic Imaging (JEI)  
To recover high-quality aerial image from its non-uniform version, we propose a novel patch-wise restoration approach based on a key observation that the degree of blurring is inevitably affected by the  ...  illuminated conditions.  ...  Most Retinex-based algorithms 35-39 extract the reflectance component as the enhanced result by isolating the illumination, and on the applications, we explore its non-local variant to correct the  ... 
doi:10.1117/1.jei.26.3.033016 fatcat:6itkkvmobfeubgs5ydtj3k2sly

Joint registration and synthesis using a probabilistic model for alignment of MRI and histological sections

Juan Eugenio Iglesias, Marc Modat, Loïc Peter, Allison Stevens, Roberto Annunziata, Tom Vercauteren, Ed Lein, Bruce Fischl, Sebastien Ourselin
2018 Medical Image Analysis  
, rather than by using mutual information (MI) as metric.  ...  We use approximate Bayesian inference to iteratively refine the probabilistic estimate of the synthesis and the registration, while accounting for each other's uncertainty.  ...  ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following:  ... 
doi:10.1016/ pmid:30282061 fatcat:gwxmcrob4zb4dk36xgqikrxjoi
« Previous Showing results 1 — 15 out of 884 results