Filters








28 Hits in 2.5 sec

ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning

Quoc V. Le, Alexandre Karpenko, Jiquan Ngiam, Andrew Y. Ng
2011 Neural Information Processing Systems  
In this paper, we propose a robust soft reconstruction cost for ICA that allows us to learn highly overcomplete sparse features even on unwhitened data.  ...  However, standard ICA requires an orthonoramlity constraint to be enforced, which makes it difficult to learn overcomplete features. In addition, ICA is sensitive to whitening.  ...  These challenges motivate our search for a better type of non-degeneracy control for ICA.  ... 
dblp:conf/nips/LeKNN11 fatcat:rkhwqg5rrncapafh4sj67gji4y

Glimpsing independent vector analysis: Separating more sources than sensors using active and inactive states

Alireza Masnadi-Shirazi, Wenyi Zhang, Bhaskar D. Rao
2010 2010 IEEE International Conference on Acoustics, Speech and Signal Processing  
In this paper, we explore the problem of separating convolutedly mixed signals in the overcomplete (degenerate) case of having more sources than sensors.  ...  This enables the algorithm to "glimpse" or listen in the gaps, hence compensating for the global degeneracy by allowing it to learn the mixing matrices at periods where it is locally less degenerate.  ...  Various methods in the past with different underlying assumptions have been proposed to deal with overcompleteness (degeneracy) in ICA linear mixing.  ... 
doi:10.1109/icassp.2010.5494905 dblp:conf/icassp/Masnadi-ShiraziZR10 fatcat:kxriizhm3neevedwqiykxa62ea

Glimpsing IVA: A Framework for Overcomplete/Complete/Undercomplete Convolutive Source Separation

Alireza Masnadi-Shirazi, Wenyi Zhang, Bhaskar D. Rao
2010 IEEE Transactions on Audio, Speech, and Language Processing  
This feature is extremely useful in dealing with overcomplete situations.  ...  Index Terms-Blind source separation (BSS), convolutive mixture, hidden Markov model (HMM), independent component analysis (ICA), independent vector analysis (IVA), overcomplete systems, speech recognition  ...  Various methods in the past with different underlying assumptions have been proposed to deal with overcompleteness (degeneracy) in ICA linear instantaneous mixing.  ... 
doi:10.1109/tasl.2010.2052609 fatcat:nzgeelzabzf7tajy5wrjequgaa

A Nonparametric Bayesian Approach toward Stacked Convolutional Independent Component Analysis

Sotirios P. Chatzis, Dimitrios Kosmopoulos
2015 2015 IEEE International Conference on Computer Vision (ICCV)  
To resolve these issues, in this paper we introduce a convolutional nonparametric Bayesian sparse ICA architecture for overcomplete feature learning from high-dimensional data.  ...  Unsupervised feature learning algorithms based on convolutional formulations of independent components analysis (ICA) have been demonstrated to yield state-of-the-art results in several action recognition  ...  In this work, we focus on unsupervised feature extractors based on stacked convolutional ICA architectures, with application to action recognition in video sequences.  ... 
doi:10.1109/iccv.2015.321 dblp:conf/iccv/ChatzisK15 fatcat:3v6gve2bfzbitdgsm6scm2bqyu

A Nonparametric Bayesian Approach Toward Stacked Convolutional Independent Component Analysis [article]

Sotirios P. Chatzis
2015 arXiv   pre-print
To resolve these issues, in this paper we introduce a convolutional nonparametric Bayesian sparse ICA architecture for overcomplete feature learning from high-dimensional data.  ...  Unsupervised feature learning algorithms based on convolutional formulations of independent components analysis (ICA) have been demonstrated to yield state-of-the-art results in several action recognition  ...  In this work, we focus on unsupervised feature extractors based on stacked convolutional ICA architectures, with application to action recognition in video sequences.  ... 
arXiv:1411.4423v5 fatcat:yrgha7mnl5aohlzgwmmx5u7f2u

Contrastive Topographic Models: Energy-based density models applied to the understanding of sensory coding and cortical topography [article]

Simon Osindero
2020 arXiv   pre-print
We apply biologically based constraints on elements of the model.  ...  vivo, whilst simultaneously learning about statistical regularities in the data.  ...  The Gibbs sampling algorithm, and some results on topographic maps also appears in co-authored with Max Welling and Geoff Hinton.  ... 
arXiv:2011.03535v1 fatcat:xqev3hqpnnglzjqs7chsrc5zoe

ON SIGNALS FAINT AND SPARSE: THE ACICA ALGORITHM FOR BLIND DE-TRENDING OF EXOPLANETARY TRANSITS WITH LOW SIGNAL-TO-NOISE

I. P. Waldmann
2013 Astrophysical Journal  
Independent Component Analysis (ICA) has recently been shown to be a promising new path in data analysis and de-trending of exoplanetary time series signals.  ...  Such approaches do not require or assume any prior or auxiliary knowledge on the data or instrument in order to de-convolve the astrophysical light curve signal from instrument or stellar systematic noise  ...  furthermore offers us an unprecedented and unique insight into the morphology of a data set by allowing us to directly map out temporal/wavelength dependent variations of instrumental or stellar noise in  ... 
doi:10.1088/0004-637x/780/1/23 fatcat:go755ynxqnhpnejzuerqfyh7uu

The local low-dimensionality of natural images [article]

Olivier J. Hénaff, Johannes Ballé, Neil C. Rabinowitz, Eero P. Simoncelli
2015 arXiv   pre-print
We develop a new statistical model for photographic images, in which the local responses of a bank of linear filters are described as jointly Gaussian, with zero mean and a covariance that varies slowly  ...  As such, this representation holds much promise for use in applications such as denoising, compression, and texture representation, and may form a useful substrate for hierarchical decompositions.  ...  ACKNOWLEDGMENTS We would like to thank Joan Bruna for helpful discussions on the use of the nuclear norm and technical aspects of the phase-recovery problem.  ... 
arXiv:1412.6626v4 fatcat:bal3xtkdx5b5vdjrwglh6kv23i

ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA [article]

Ilyes Khemakhem, Ricardo Pio Monti, Diederik P. Kingma, Aapo Hyvärinen
2020 arXiv   pre-print
Our results extend recent developments in nonlinear ICA, and in fact, they lead to an important generalization of ICA models.  ...  In our model family, the energy function is the dot-product between two feature extractors, one for the dependent variable, and one for the conditioning variable.  ...  In addition, using perfectly identifiable networks in real life applications eliminates the randomness and arbitrariness of the system, and gives more control to the operator.  ... 
arXiv:2002.11537v4 fatcat:4tn4lmmoqbhu3eaag7323wifuq

10.1162/jmlr.2003.4.7-8.1499

2000 Applied Physics Letters  
This paper presents a new ICA related probabilistic model where the number of sources exceeds the number of sensors to perform an image segmentation and illumination removal, simultaneously.  ...  It is well known that even slight changes in nonuniform illumination lead to a large image variability and are crucial for many visual tasks.  ...  As has already been discussed in the Introduction, strong constraints on the nature of the sources need to be imposed to solve the ICA problem in the ill-posed case.  ... 
doi:10.1162/jmlr.2003.4.7-8.1499 fatcat:mdsjxymhone4hmki35u5j6u3hu

Dictionary Learning and Tensor Decomposition via the Sum-of-Squares Method [article]

Boaz Barak, Jonathan A. Kelner, David Steurer
2014 arXiv   pre-print
T' that is τ-close to T in the spectral norm (when considered as a matrix).  ...  For the case m=O(n), our algorithm recovers every column of A within arbitrarily good constant accuracy in time m^O( m/(τ^-1)), in particular achieving polynomial time if τ = m^-δ for any δ>0, and time  ...  Independent Component Analysis (ICA) [Com94] is one method that can be used for the dictionary learning in the case the random variables x 1 , . . . , x n are statistically independent.  ... 
arXiv:1407.1543v2 fatcat:mup4et5bibgj5dhl6oozjpcdje

Polyphonic Transcription By Non-Negative Sparse Coding Of Power Spectra

Samer M. Abdallah, Mark D. Plumbley
2004 Zenodo  
The algorithm was based on the overcomplete, noisy ICA methods of [13] .  ...  ), we use an adaptive prior in the form of an explicit ICA model.  ... 
doi:10.5281/zenodo.1415071 fatcat:jm3ixoiiz5bwxi4ugesfh2yali

Dictionary Learning and Tensor Decomposition via the Sum-of-Squares Method

Boaz Barak, Jonathan A. Kelner, David Steurer
2015 Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing - STOC '15  
T that is τ-close to T in the spectral norm (when considered as a matrix).  ...  For the case m = O(n), our algorithm recovers every column of A within arbitrarily good constant accuracy in time m O(log m/ log(τ −1 )) , in particular achieving polynomial time if τ = m −δ for any δ  ...  Conclusions and Open Problems The Sum of Squares method has found many uses across a variety of disciplines, and in this work we demonstrate its potential for solving unsupervised learning problems in  ... 
doi:10.1145/2746539.2746605 dblp:conf/stoc/BarakKS15 fatcat:7cdrj3iz7bgvvhklltjk6zbliy

OF "COCKTAIL PARTIES" AND EXOPLANETS

I. P. Waldmann
2012 Astrophysical Journal  
Correcting the raw data at the 10^-4 level of accuracy in flux is one of the central challenges.  ...  Such a 'blind' signal de-mixing is commonly known as the 'Cocktail Party problem' in signal-processing.  ...  However, ICA is limited by two degeneracies: 1) Maximising the non-Gaussianity of components in order to obtain mutual independence dictates that only a maximum of one component may have a Gaussian probability  ... 
doi:10.1088/0004-637x/747/1/12 fatcat:bajm5u477zdpbftznoozdpu66i

Learning Informative Features from Restricted Boltzmann Machines

Jakub M. Tomczak
2015 Neural Processing Letters  
In order to verify our hypothesis we apply the regularization term to the Restricted Boltzmann Machine (RBM) and carry out empirical study on three classification problems: character recognition, object  ...  One hypothesis focuses on formulating good criterion (prior) that may help to learn a set of features capable of disentangling hidden factors.  ...  This observation served as a starting point for the authors of [22] to formulate a new regularizer for the non-degeneracy control of the weights matrix: Ω rc (θ ) = 1 2N N n=1 W Wx n − x n 2 2 . (17)  ... 
doi:10.1007/s11063-015-9491-9 fatcat:2k3afdc7wzbs3ktmz4shmfkfye
« Previous Showing results 1 — 15 out of 28 results