10,554 Hits in 3.0 sec

Information Bottleneck on General Alphabets

George Pichler, Gunther Koliander
2018 2018 IEEE International Symposium on Information Theory (ISIT)  
We prove rigorously a source coding theorem that can probably be considered folklore, a generalization to arbitrary alphabets of a problem motivated by the Information Bottleneck method.  ...  The proof relies on the well established discrete case and showcases a technique for lifting discrete coding theorems to arbitrary alphabets.  ...  However, we will rely on the discrete case (Lem. 14) and showcase a technique to lift it to general alphabets. Remark 4.  ... 
doi:10.1109/isit.2018.8437714 dblp:conf/isit/PichlerK18 fatcat:uzweyexepngl3ge7cgwjqw2ph4

Information Bottleneck on General Alphabets [article]

Georg Pichler, Günther Koliander
2018 arXiv   pre-print
We prove rigorously a source coding theorem that can probably be considered folklore, a generalization to arbitrary alphabets of a problem motivated by the Information Bottleneck method.  ...  The proof relies on the well established discrete case and showcases a technique for lifting discrete coding theorems to arbitrary alphabets.  ...  This provides a single-letter solution to the information-theoretic problem behind the information bottleneck method for arbitrary random sources and in particular it shows, that the information bottleneck  ... 
arXiv:1801.01050v2 fatcat:lnwz7mv4evhlpkhsn5osamywte

Associative Memory with Pattern Analysis and Synthesis by a Bottleneck Neural Network(Contribution to 21 Century Intelligent Technologies and Bioinformatics)

Eiichi INOHIRA, Takeshi OGAWA, Hirokazu YOKOI
2008 International journal of biomedical soft computing and human sciences  
These outputs include informatien on synthesis of a key pattern from compressed information in the middle layer.  ...  One is to add intermediate elements into MLP to improve its perfbmmce. The other is to use outputs of hidden layers in a five-layer bottleneck neural network.  ...  bottleneck neural network but also analyzedlsynthesized information facilitates heteroassociation and increases recall perfbrmance because analyzedrsynthesized information has less effect on  ... 
doi:10.24466/ijbschs.13.2_27 fatcat:zgsfx2xjlzdfxpekffxsf7c5pa

Coarsely Quantized Decoding and Construction of Polar Codes Using the Information Bottleneck Method

Shah, Stark, Bauch
2019 Algorithms  
The information bottleneck method is a generic clustering framework from the fieldof machine learning which allows compressing an observed quantity while retaining as much ofthe mutual information it shares  ...  This paper presents, in detail, the application of the information bottleneck method to polarcodes, where the framework is used to compress the virtual bit channels defined in the code structureand show  ...  Information Bottleneck Method The information bottleneck method [3, 4] is a generic clustering framework with its roots in the field of machine learning.  ... 
doi:10.3390/a12090192 fatcat:whxc542dmbat3e4klyhnrfzl74

Optimal Kullback–Leibler Aggregation via Information Bottleneck

Bernhard C. Geiger, Tatjana Petrov, Gernot Kubin, Heinz Koeppl
2015 IEEE Transactions on Automatic Control  
Then, we express the problem in the form of information bottleneck optimization, and propose using the agglomerative information bottleneck algorithm for searching a sub-optimal partition greedily, rather  ...  The theory is illustrated with examples and one application scenario in the context of modeling bio-molecular interactions.  ...  Let S be another RV with alphabet S representing relevant information. The information loss relevant w.r.t.  ... 
doi:10.1109/tac.2014.2364971 fatcat:sdvejzbrdjfyjiwx2yi4pdnjii

Information Bottleneck Decoding of Rate-Compatible 5G-LDPC Codes [article]

Maximilian Stark, Linfang Wang, Richard D. Wesel, Gerhard Bauch
2019 arXiv   pre-print
Recently, the information bottleneck (IB) method was used to design mutual-information-maximizing lookup tables that replace conventional finite-precision node computations.  ...  As an example of the new approach, a 4-bit information bottleneck decoder is evaluated for PBRL LDPC codes over a typical range of rates.  ...  Without loss of generality, we consider a variable node with degree d v = 4 which processes one channel message and three messages received from connected check nodes to generate extrinsic information  ... 
arXiv:1906.08985v2 fatcat:tkdgtoxwv5hlpb2avs2zoi3qj4

Information-Bottleneck Decoding of High-Rate Irregular LDPC Codes for Optical Communication Using Message Alignment

Maximilian Stark, Jan Lewandowsky, Gerhard Bauch
2018 Applied Sciences  
Recently, so-called information-bottleneck decoders were proposed which leverage a machine learning framework (i.e., the information bottleneck method) to design coarse-precision decoders with error-correction  ...  In this paper, we show that the complexity of information-bottleneck decoders remains manageable for irregular LDPC codes if our proposed construction approach is deployed.  ...  Please note that, assuming a trivial straightforward generalization of information bottleneck decoders to irregular LDPC codes, one would stop at this point.  ... 
doi:10.3390/app8101884 fatcat:4mmvdf2htjchljadwwgp2raena

Discrete profile alignment via constrained information bottleneck

Sean O'Rourke, Gal Chechik, Robin Friedman, Eleazar Eskin
2004 Neural Information Processing Systems  
By using an extension of information bottleneck (IB) incorporating constraints and priors on the class distributions, we find an informationally optimal alphabet.  ...  This discretization yields a concise, informative textual representation for profile sequences.  ...  This is achieved using a variant of Information Bottleneck (IB) [9] , a distributional clustering approach for informationally optimal discretization.  ... 
dblp:conf/nips/ORourkeCFE04 fatcat:46r2u57mkzal3kcz3d2mmfcdvm

Decoding of Non-Binary LDPC Codes Using the Information Bottleneck Method [article]

Maximilian Stark, Jan Lewandowsky, Souradip Saha, Gerhard Bauch
2019 arXiv   pre-print
A machine learning framework termed the information bottleneck method is used to design the corresponding lookup tables.  ...  In this approach, mutual-information maximizing lookup tables replace the conventional operations of the variable nodes and the check nodes in message passing decoding.  ...  Please note that the message y in 0 for c 0 is excluded since extrinsic information on c 0 shall be generated.  ... 
arXiv:1810.08921v2 fatcat:sigmeybv3jh7riyqkohjsrc5vu

Deformed Statistics Formulation of the Information Bottleneck Method [article]

R. C. Venkatesan, A. Plastino
2009 arXiv   pre-print
The theoretical basis for a candidate variational principle for the information bottleneck (IB) method is formulated within the ambit of the generalized nonadditive statistics of Tsallis.  ...  Defining X , X̃ , and Y to be the source alphabet, the compressed reproduction alphabet, and, the relevance variable respectively, it is demonstrated that minimization of a generalized IB (gIB) Lagrangian  ...  GENERALIZED INFORMATION BOTTLENECK VARIATIONAL PRINCIPLE A.  ... 
arXiv:0811.3174v4 fatcat:vc3hlphvxndklncmcaajibqvaa

Graph Entropy, Network Coding and Guessing games [article]

Soren Riis
2007 arXiv   pre-print
Shannon's information inequalities can be used to calculate upper bounds on a graph's entropy as well as calculating the size of the minimal index code.  ...  Based on this result we show that there exist graphs with an entropy that cannot be calculated using only Shannon's classical information inequalities, and show that better estimate can be obtained by  ...  Introduction Main results Informally, one of the key problems in proving lower bounds on circuits is to identify and prove that certain Circuit topologies (e.g. small circuits) provide information bottlenecks  ... 
arXiv:0711.4175v1 fatcat:i5vuxg2e5nbrpkuwxsyfay6vnu

Generating Chinese Classical Poems with RNN Encoder-Decoder [chapter]

Xiaoyuan Yi, Ruoyu Li, Maosong Sun
2017 Lecture Notes in Computer Science  
This paper examines the generalization abilities of encoder-decoder networks on a class of subregular functions characteristic of natural language reduplication.  ...  must pass all information through the encoder bottleneck.  ...  Figure 5 : 5 Effect on varying alphabet size and maximum string length, with minumum string length fixed at 3, on generalization accuracy.  ... 
doi:10.1007/978-3-319-69005-6_18 fatcat:o6ipo2tecrg6tpvvo76nocvqzu

The information science of microbial ecology

Aria S Hahn, Kishori M Konwar, Stilianos Louca, Niels W Hanson, Steven J Hallam
2016 Current Opinion in Microbiology  
to eliminate bottlenecks and empower knowledge creation and translation.  ...  A revolution is unfolding in microbial ecology where petabytes of 'multi-omics' data are produced using next generation sequencing and mass spectrometry platforms.  ...  'alphabet'.  ... 
doi:10.1016/j.mib.2016.04.014 pmid:27183115 fatcat:bhlgmvfmk5gsflhb6dzkg7jg2i

Online Suffix Tree Construction for Streaming Sequences [chapter]

Giyasettin Ozcan, Adil Alpkocak
2008 Communications in Computer and Information Science  
Even more, performance of the construction suffers when alphabet size is large.  ...  Experimental results showed that our approach outperforms existing ones.  ...  In 1830's, Morse alphabet is inspired by such information. Similarly learning such information from a domain expert can enhance suffix tree construction for music sequences.  ... 
doi:10.1007/978-3-540-89985-3_9 fatcat:orkbi5h6nrbufpv47tnox4noly

A Submodularity-based Agglomerative Clustering Algorithm for the Privacy Funnel [article]

Ni Ding, Parastoo Sadeghi
2019 arXiv   pre-print
We show that the IAC-MDSF algorithm also applies to the information bottleneck (IB), a dual problem to PF.  ...  For a data curator that wants to share the data X correlated with the sensitive information S, the PF problem is to generate the sanitized data X̂ that maintains a specified utility/fidelity threshold  ...  Information Bottleneck Problem The duality relationship between PF and information bottleneck (IB) has been pointed out in [4] , [12] .  ... 
arXiv:1901.06629v2 fatcat:r355dwp75bfz5mrmd7ofn7rlmq
« Previous Showing results 1 — 15 out of 10,554 results