Filters








10,157 Hits in 4.5 sec

Stochastic Mutual Information Gradient Estimation for Dimensionality Reduction Networks

Ozan Ozdenizci, Deniz Erdogmus
<span title="2021-05-01">2021</span>
We present a dimensionality reduction network (MMINet) training procedure based on the stochastic estimate of the mutual information gradient.  ...  The network projects high-dimensional features onto an output feature space where lower dimensional representations of features carry maximum mutual information with their associated class labels.  ...  Discussion We present a supervised dimensionality reduction network training procedure based on the stochastic estimate of the mutual information gradient.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.48550/arxiv.2105.00191">doi:10.48550/arxiv.2105.00191</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/j25kc727ajfy3okox3q7nrmxea">fatcat:j25kc727ajfy3okox3q7nrmxea</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20220511140301/https://arxiv.org/pdf/2105.00191.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/9f/ab/9fab36608c0aa634fe2f2901429f7ba3158f5ae6.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.48550/arxiv.2105.00191"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> Publisher / doi.org </button> </a>

Hindsight Value Function for Variance Reduction in Stochastic Dynamic Environment [article]

Jiaming Guo, Rui Zhang, Xishan Zhang, Shaohui Peng, Qi Yi, Zidong Du, Xing Hu, Qi Guo, Yunji Chen
<span title="2021-08-05">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
In this paper, we propose to replace the state value function with a novel hindsight value function, which leverages the information from the future to reduce the variance of the gradient estimate for  ...  Particularly, to obtain an ideally unbiased gradient estimate, we propose an information-theoretic approach, which optimizes the embeddings of the future to be independent of previous actions.  ...  L F (θ f ) is an estimate of the upper bound of mutual information. It is supposed to optimize the gradient estimate towards bias-free.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2107.12216v2">arXiv:2107.12216v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/vygbpu2ybbhwrdf22l4jkm27ti">fatcat:vygbpu2ybbhwrdf22l4jkm27ti</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210807045413/https://arxiv.org/pdf/2107.12216v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/46/51/46515eb6eb66973db6bd9cef146853badb938b7e.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2107.12216v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

A Hybrid Gradient Method to Designing Bayesian Experiments for Implicit Models [article]

Jiaxin Zhang, Sirui Bi, Guannan Zhang
<span title="2021-03-14">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
In this work, we propose a hybrid gradient approach that leverages recent advances in variational MI estimator and evolution strategies (ES) combined with black-box stochastic gradient ascent (SGA) to  ...  The optimal design is usually achieved by maximizing the mutual information (MI) between the data and the model parameters.  ...  Estimating and optimizing MI is core to many machine learning research but it has been a challenge to bounding MI in high dimensions. Mutual information estimators Belghazi et al.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2103.08594v1">arXiv:2103.08594v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/7jkapaxzqjavtecru5t6jrb2me">fatcat:7jkapaxzqjavtecru5t6jrb2me</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210324030927/https://arxiv.org/pdf/2103.08594v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/29/d9/29d971379fc9013c8a06c9862b473f78d6d7c649.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2103.08594v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

A Scalable Gradient-Free Method for Bayesian Experimental Design with Implicit Models [article]

Jiaxin Zhang, Sirui Bi, Guannan Zhang
<span title="2021-03-14">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
In this paper, we propose a novel approach that leverages recent advances in stochastic approximate gradient ascent incorporated with a smoothed variational MI estimator for efficient and robust BED.  ...  For implicit models, where the likelihood is intractable but sampling is possible, conventional BED methods have difficulties in efficiently estimating the posterior distribution and maximizing the mutual  ...  Mutual Information Estimation Mutual information (MI) estimation plays a critical role in many important problems, not only the BED framework but also other machine learning tasks such as reinforcement  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2103.08026v1">arXiv:2103.08026v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/jksb3ia75vfbdn3xakpvvqy2ma">fatcat:jksb3ia75vfbdn3xakpvvqy2ma</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210325185949/https://arxiv.org/pdf/2103.08026v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/05/d5/05d5e04f1c3c03e77d6147fc4f258de4da30a248.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2103.08026v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Information Theoretic Feature Transformation Learning for Brain Interfaces

Ozan Ozdenizci, Deniz Erdogmus
<span title="2019-03-28">2019</span> <i title="Institute of Electrical and Electronics Engineers (IEEE)"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/nrcoa2vuhjcvfctty6zgus57um" style="color: black;">IEEE Transactions on Biomedical Engineering</a> </i> &nbsp;
A variety of pattern analysis techniques for model training in brain interfaces exploit neural feature dimensionality reduction based on feature ranking and selection heuristics.  ...  Exploiting the state-of-the-art methods for initial feature vector construction, we compare the proposed approaches with conventional feature selection-based dimensionality reduction techniques, which  ...  This work addresses the potential confounders caused by heuristic feature ranking and selection based dimensionality reduction methods that are widely used for brain interfaces.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/tbme.2019.2908099">doi:10.1109/tbme.2019.2908099</a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pubmed/30932828">pmid:30932828</a> <a target="_blank" rel="external noopener" href="https://pubmed.ncbi.nlm.nih.gov/PMC7008579/">pmcid:PMC7008579</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/6jyybrrhe5eyjougtk7h3oniry">fatcat:6jyybrrhe5eyjougtk7h3oniry</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200915142434/https://arxiv.org/pdf/1903.12235v1.pdf" title="fulltext PDF download [not primary version]" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <span style="color: #f43e3e;">&#10033;</span> <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/35/a9/35a985a7b0b34b7513b72e8238775e76008baa6c.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/tbme.2019.2908099"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7008579" title="pubmed link"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> pubmed.gov </button> </a>

Informative Neural Ensemble Kalman Learning [article]

Margaret Trautner and Gabriel Margolis and Sai Ravela
<span title="2020-08-22">2020</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Neural Learning also embodies stochastic dynamics, but informative Learning is less developed.  ...  In stochastic systems, informative approaches select key measurement or decision variables that maximize information gain to enhance the efficacy of model-related inferences.  ...  ACKNOWLEDGEMENTS The authors thank the Earth Signals and Systems Group, MIT, and anonymous reviewers for constructive reviews.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2008.09915v1">arXiv:2008.09915v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/ev4rc7av75dxhfl2nyrn43h6dm">fatcat:ev4rc7av75dxhfl2nyrn43h6dm</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200831200322/https://arxiv.org/pdf/2008.09915v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2008.09915v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Multi-Task Learning Using Gradient Balance and Clipping with an Application in Joint Disparity Estimation and Semantic Segmentation

Yiyou Guo, Chao Wei
<span title="2022-04-12">2022</span> <i title="MDPI AG"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/ikdpfme5h5egvnwtvvtjrnntyy" style="color: black;">Electronics</a> </i> &nbsp;
Specifically, we introduce a multi-task stochastic gradient descent optimization (MTSGD) to learn task-specific and shared representation in the deep neural network.  ...  Meanwhile, we perform a series of ablation studies to have a deep analysis of gradient descent for MTL.  ...  Acknowledgments: The authors would like to thank the anonymous reviewers for their constructive comments and suggestions, which strengthened this paper a lot.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.3390/electronics11081217">doi:10.3390/electronics11081217</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/zfqzxo4x75exridfxklqillniu">fatcat:zfqzxo4x75exridfxklqillniu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20220429022251/https://mdpi-res.com/d_attachment/electronics/electronics-11-01217/article_deploy/electronics-11-01217.pdf?version=1649749957" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/65/84/658473fc1ff2b898812a032d2e3c6b21758ad404.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.3390/electronics11081217"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> mdpi.com </button> </a>

Adaptive motion estimation schemes using maximum mutual information criterion

Jing Zhao, Dapeng Wu, Deniz Erdogmus, Yuguang Fang, Zhihai He
<span title="">2007</span> <i title="Wiley"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/6o4hgxplrbehxg4t53ub7zmfha" style="color: black;">Wireless Communications and Mobile Computing</a> </i> &nbsp;
In this paper, we develop an adaptive system under the criterion of maximum mutual information to address the motion estimation problem.  ...  is used for motion estimation.  ...  By utilizing the same complexity reduction techniques to mutual information entropy of N samples of random variable [x, y] T , a non-parametric stochastic estimator for mutual information is obtained:  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1002/wcm.474">doi:10.1002/wcm.474</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/5lefzyzn45hnljdm54udgjaxvy">fatcat:5lefzyzn45hnljdm54udgjaxvy</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20100609062959/http://www.fang.ece.ufl.edu/mypaper/wcmc07zhao.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/31/7a/317aa7401521e93c8778af02572039754476ce92.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1002/wcm.474"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> wiley.com </button> </a>

Stochastic Curiosity Exploration for Dialogue Systems

Jen-Tzung Chien, Po-Chien Hsu
<span title="2020-10-25">2020</span> <i title="ISCA"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/trpytsxgozamtbp7emuvz2ypra" style="color: black;">Interspeech 2020</a> </i> &nbsp;
This agent encourages the exploration for future diversity based on a latent dynamic architecture which consists of encoder network, curiosity network, information network and policy network.  ...  The latent states and actions are drawn to predict stochastic transition for future.  ...  However, it is challenging to construct a neural estimation for mutual information. An analytical neural network solution to deep RL based on SCE is required.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.21437/interspeech.2020-1313">doi:10.21437/interspeech.2020-1313</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/interspeech/ChienH20a.html">dblp:conf/interspeech/ChienH20a</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/3ceyifmeqnhtpebkoagobusohm">fatcat:3ceyifmeqnhtpebkoagobusohm</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20201211084300/https://www.isca-speech.org/archive/Interspeech_2020/pdfs/1313.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/3f/f8/3ff805468653bee38de2103ec487b7be49fde1ca.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.21437/interspeech.2020-1313"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

Bayesian Experimental Design for Implicit Models by Mutual Information Neural Estimation [article]

Steven Kleinegesse, Michael U. Gutmann
<span title="2020-08-14">2020</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
The field of Bayesian experimental design advocates that, ideally, we should choose designs that maximise the mutual information (MI) between the data and the parameters.  ...  In this paper, we propose a new approach to Bayesian experimental design for implicit models that leverages recent advances in neural MI estimation to deal with these issues.  ...  Acknowledgements Steven Kleinegesse was supported in part by the EPSRC Centre for Doctoral Training in Data Science, funded by the UK Engineering and Physical Sciences Research Council (grant EP/L016427  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2002.08129v3">arXiv:2002.08129v3</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/eifjybowbvc6rejyfoexheucm4">fatcat:eifjybowbvc6rejyfoexheucm4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200929212915/https://arxiv.org/pdf/2002.08129v3.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/1a/ea/1aea4d77fc7a4a79a858084069229867bc37d81e.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2002.08129v3" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Opening the Black Box of Deep Neural Networks via Information [article]

Ravid Shwartz-Ziv, Naftali Tishby
<span title="2017-04-29">2017</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
They suggested that the goal of the network is to optimize the Information Bottleneck (IB) tradeoff between compression and prediction, successively, for each layer.  ...  (ii) The representation compression phase begins when the training errors becomes small and the Stochastic Gradient Decent (SGD) epochs change from a fast drift to smaller training error into a stochastic  ...  Acknowledgements This study was supported by the Israeli Science Foundation center of excellence, the Intel Collaborative Research Institute for Computational Intelligence (ICRI-CI), and the Gatsby Charitable  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1703.00810v3">arXiv:1703.00810v3</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/qjh25u2ztjc7tfgkr7oqehqr5i">fatcat:qjh25u2ztjc7tfgkr7oqehqr5i</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200823183953/https://arxiv.org/pdf/1703.00810v3.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/0b/30/0b304b127a9290d37d83e2f32d35e045bbb8d339.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1703.00810v3" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

A stochastic model for natural feature representation

S. Kumar, F. Ramos, B. Upcroft, M. Ridley, L. Ong, S. Sakkarieh, H. Durrant-Whyte
<span title="">2005</span> <i title="IEEE"> 2005 7th International Conference on Information Fusion </i> &nbsp;
This paper presents a robust stochastic model for the incorporation of natural features within data fusion algorithms.  ...  The resulting compactness of the representation is especially suitable to decentralized sensor networks.  ...  The stochastic estimate is versatile as the inferred covariances quantify the uncertainty in the visual state estimation.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/icif.2005.1591971">doi:10.1109/icif.2005.1591971</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/2fhrg2lhjjgnhe4qp5bontslyi">fatcat:2fhrg2lhjjgnhe4qp5bontslyi</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20171108135102/https://core.ac.uk/download/pdf/10902864.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/dd/83/dd833a78814ec48ecf86c34cc8386f85dd327356.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/icif.2005.1591971"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>

An Information Theoretic Framework for Eukaryotic Gradient Sensing [chapter]

<span title="">2007</span> <i title="The MIT Press"> Advances in Neural Information Processing Systems 19 </i> &nbsp;
Adapting a method for estimation of spike train entropies described by Victor (originally due to Kozachenko and Leonenko), we estimate lower bounds on the mutual information between the transmitted signal  ...  We show that the time course of the mutual information between the cell's surface receptors and the (unknown) gradient direction is consistent with experimentally measured cellular response times.  ...  Direct estimate of the mutual information from stochastic simulations is impractical because the aggregate random variables occupy a 980 dimensional space that a limited number of simulation runs cannot  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.7551/mitpress/7503.003.0093">doi:10.7551/mitpress/7503.003.0093</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/icb6tet6knbj3i3o4kouiaycnm">fatcat:icb6tet6knbj3i3o4kouiaycnm</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20160411064400/http://papers.nips.cc/paper/2995-an-information-theoretic-framework-for-eukaryotic-gradient-sensing.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/7e/0d/7e0df41e4bab62de00fcfa5bf42c2f5033f58411.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.7551/mitpress/7503.003.0093"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

Information Bottleneck Analysis by a Conditional Mutual Information Bound

Taro Tezuka, Shizuma Namekawa
<span title="2021-07-29">2021</span> <i title="MDPI AG"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/4d3elkqvznfzho6ki7a35bt47u" style="color: black;">Entropy</a> </i> &nbsp;
We used mutual information neural estimation (MINE) to estimate I(z;x|y).  ...  We extend this framework by demonstrating that conditional mutual information I(z;x|y) provides an alternative upper bound for I(z;n).  ...  Estimation Estimating mutual information for random variables with unknown distributions is a challenging task. It is even more so for high-dimensional random variables.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.3390/e23080974">doi:10.3390/e23080974</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/ruztsjaibngtfblxqqjtiq4of4">fatcat:ruztsjaibngtfblxqqjtiq4of4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210731033906/https://res.mdpi.com/d_attachment/entropy/entropy-23-00974/article_deploy/entropy-23-00974-v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/24/a8/24a8fab69d79eb3271e6048ae9d749c9a530dae2.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.3390/e23080974"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> mdpi.com </button> </a>

Specializing Word Embeddings (for Parsing) by Information Bottleneck

Xiang Lisa Li, Jason Eisner
<span title="">2019</span> <i title="Association for Computational Linguistics"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/u3ideoxy4fghvbsstiknuweth4" style="color: black;">Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)</a> </i> &nbsp;
In the continuous version, we show experimentally that moderately compressing the word embeddings by our method yields a more accurate parser in 8 of 9 languages, unlike simple dimensionality reduction  ...  We propose a very fast variational information bottleneck (VIB) method to nonlinearly compress these embeddings, keeping only the information that helps a discriminative parser.  ...  These stochastic vectors yield improved parsing results, in a way that simpler dimensionality reduction methods do not.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.18653/v1/d19-1276">doi:10.18653/v1/d19-1276</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/emnlp/LiE19.html">dblp:conf/emnlp/LiE19</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/7jg4gu7bbvc4jpwsz33iu7ucfu">fatcat:7jg4gu7bbvc4jpwsz33iu7ucfu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20191116064855/https://www.aclweb.org/anthology/D19-1276.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/52/16/52168908b321bf1312381e88e6566c450caaf100.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.18653/v1/d19-1276"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>
&laquo; Previous Showing results 1 &mdash; 15 out of 10,157 results