Filters








4,902 Hits in 4.2 sec

Generative Maximum Entropy Learning for Multiclass Classification [article]

Ambedkar Dukkipati, Gaurav Pandey, Debarghya Ghoshdastidar, Paramita Koley, D. M. V. Satya Sriram
2013 arXiv   pre-print
In this paper, we introduce a maximum entropy classification method with feature selection for large dimensional data such as text datasets that is generative in nature.  ...  For two class problems, in the proposed method, we use Jeffreys (J) divergence to discriminate the class conditional densities.  ...  For text classification, a simple discriminative maximum entropy technique was used in [7] to estimate the posterior distribution of class variable conditioned on training data.  ... 
arXiv:1205.0651v3 fatcat:hlv2qadugrg6xfktejwq4kr7ay

Generative Maximum Entropy Learning for Multiclass Classification

Ambedkar Dukkipati, Gaurav Pandey, Debarghya Ghoshdastidar, Paramita Koley, D.M.V. Satya Sriram
2013 2013 IEEE 13th International Conference on Data Mining  
In this paper, we introduce a maximum entropy classification method with feature selection for large dimensional data such as text datasets that is generative in nature.  ...  For two class problems, in the proposed method, we use Jeffreys (J) divergence to discriminate the class conditional densities.  ...  For text classification, a simple discriminative maximum entropy technique was used in [7] to estimate the posterior distribution of class variable conditioned on training data.  ... 
doi:10.1109/icdm.2013.26 dblp:conf/icdm/DukkipatiPGKS13 fatcat:2emo6jy7yvbrtcolim6inqingq

A Robust Discriminative Term Weighting Based Linear Discriminant Method for Text Classification

Khurum Nazir Junejo, Asim Karim
2008 2008 Eighth IEEE International Conference on Data Mining  
We present a supervised text classification method based on discriminative term weighting, discrimination information pooling, and linear discrimination.  ...  Text classification is widely used in applications ranging from e-mail filtering to review classification.  ...  Acknowledgment The first author gratefully acknowledges support from Lahore University of Management Sciences (LUMS) and Higher Education Commission (HEC) of Pakistan for this research.  ... 
doi:10.1109/icdm.2008.26 dblp:conf/icdm/JunejoK08 fatcat:snvyz7zv4rduhiobfvobtzdy6e

Toward Optimal Feature Selection in Naive Bayes for Text Categorization

Bo Tang, Steven Kay, Haibo He
2016 IEEE Transactions on Knowledge and Data Engineering  
Based on the JMH-divergence, we develop two efficient feature selection methods, termed maximum discrimination (MD) and MD-χ^2 methods, for text categorization.  ...  In this paper, we present a novel and efficient feature selection framework based on the Information Theory, which aims to rank the features with their discriminative capacity for classification.  ...  Hence, it is natural to select those features that have the maximum discriminative capacity for classification, by minimizing the classification error (i.e., maximizing the KL-divergence or the J-divergence  ... 
doi:10.1109/tkde.2016.2563436 fatcat:v7a64udu3nf7hkwch5wql6ldfm

FSMJ: Feature Selection with Maximum Jensen-Shannon Divergence for Text Categorization [article]

Bo Tang, Haibo He
2016 arXiv   pre-print
In this paper, we present a new wrapper feature selection approach based on Jensen-Shannon (JS) divergence, termed feature selection with maximum JS-divergence (FSMJ), for text categorization.  ...  We conduct several experiments on real-life data sets, compared with the state-of-the-art feature selection approaches for text categorization.  ...  maximum JS-divergence (FSMJ) to greedily find the most discriminative features for multi-class classification.  ... 
arXiv:1606.06366v1 fatcat:ruz5e22ggfe3zobejdlmyomd4q

Image Retrieval and Annotation Using Maximum Entropy [chapter]

Thomas Deselaers, Tobias Weyand, Hermann Ney
2007 Lecture Notes in Computer Science  
Furthermore the maximum entropy approach is used for the automatic image annotation tasks in combination with a part-based object model.  ...  using the maximum entropy framework.  ...  The best two results of 77.3% and 80.2% were achieved with our discriminative classification method.  ... 
doi:10.1007/978-3-540-74999-8_91 fatcat:4hk4kdhygnhhrl74hqqtgsmawi

Maximum Entropy Regularization and Chinese Text Recognition [article]

Changxu Cheng, Wuheng Xu, Xiang Bai, Bin Feng, Wenyu Liu
2020 arXiv   pre-print
We propose to apply Maximum Entropy Regularization to regularize the training process, which is to simply add a negative entropy term to the canonical cross-entropy loss without any additional parameters  ...  Experiments on Chinese character recognition, Chinese text line recognition and fine-grained image classification achieve consistent improvement, proving that the regularization is beneficial to generalization  ...  The maximum entropy principle [12] also points out that the model with the largest entropy of output distribution can represent features best.  ... 
arXiv:2007.04651v1 fatcat:3tahtsd7w5gpjehtjwnnsw6gku

Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval

Wei Chen, Yu Liu, Erwin M. Bakker, Michael S. Lew
2021 Pattern Recognition  
For this purpose, a modality classifier (as a discriminator) is built to distinguish the text and image modalities according to their different statistical properties.  ...  This discriminator uses its output probabilities to compute Shannon information entropy, which measures the uncertainty of the modality classification it performs.  ...  We would like to thank Theodoros Georgiou and Nan Pu for several discussions and thank NVIDIA for the donation of GPU cards.  ... 
doi:10.1016/j.patcog.2021.107983 fatcat:jzx5h5yfzvhvphpzdycowd5c24

Maximum Entropy Learning with Deep Belief Networks

Payton Lin, Szu-Wei Fu, Syu-Siang Wang, Ying-Hui Lai, Yu Tsao
2016 Entropy  
We present a maximum entropy (ME) learning algorithm for DBNs, designed specifically to handle limited training data.  ...  Results of text classification and object recognition tasks demonstrate ME-trained DBN outperforms ML-trained DBN when training data is limited.  ...  Table 1 . 1 Classification error rates (in%) for DBNs with Maximum Likelihood (ML-DBN), Maximum Entropy (ME-DBN), and Constrained Maximum Entropy (CME-DBN) trained on 5%, 10%, 15%, and 20% of training  ... 
doi:10.3390/e18070251 fatcat:xgfit3w5rvdvdhnrr7bm6ngi5i

Simple, robust, scalable semi-supervised learning via expectation regularization

Gideon S. Mann, Andrew McCallum
2007 Proceedings of the 24th international conference on Machine learning - ICML '07  
with an additional term that encourages model predictions on unlabeled data to match certain expectations-such as label priors.  ...  This paper presents expectation regularization, a semi-supervised learning method for exponential family parametric models that augments the traditional conditional label-likelihood objective function  ...  Across all of the experiments we compare with supervised naïve Bayes and maximum entropy models, and semi-supervised naïve Bayes trained with EM and maximum entropy models trained with entropy regularization  ... 
doi:10.1145/1273496.1273571 dblp:conf/icml/MannM07 fatcat:44uc34pbxzgendhzc2yreo5tle

Feature Selection and Dualities in Maximum Entropy Discrimination [article]

Tony S. Jebara, Tommi S. Jaakkola
2013 arXiv   pre-print
The feature selection method is developed as an extension to the recently proposed maximum entropy discrimination (MED) framework.  ...  In this paper we formalize feature selection specifically from a discriminative perspective of improving classification/regression accuracy.  ...  We wish to solve for a distribution of parameters of a discriminative regres sion function as well as margin variables: Theorem 3 The maximum entropy discrimination regression problem can be cast as  ... 
arXiv:1301.3865v1 fatcat:h6eumv4sbvcprg4jw7cxr6n7tm

Divergence measures for statistical data processing—An annotated bibliography

Michèle Basseville
2013 Signal Processing  
This note provides a bibliography of investigations based on or related to divergence measures for theoretical and applied inference problems.  ...  Mesures de distance pour le traitement statistique de données Résumé : Cette note contient une bibliographie de travaux concernant l'utilisation de divergences dans des problèmes relatifsà l'inférence  ...  ) This has been applied to word clustering for text classification [75] .  ... 
doi:10.1016/j.sigpro.2012.09.003 fatcat:i5ki4ziujvf7hawvj663cqqzcu

Maximum-Entropy Fine-Grained Classification [article]

Abhimanyu Dubey, Otkrist Gupta, Ramesh Raskar, Nikhil Naik
2018 arXiv   pre-print
Utilizing this notion of small visual diversity, we revisit Maximum-Entropy learning in the context of fine-grained classification, and provide a training routine that maximizes the entropy of the output  ...  probability distribution for training convolutional neural networks on FGVC tasks.  ...  Maximum-Entropy classification also improves prediction performance for CNN architectures specifically designed for fine-grained visual classification.  ... 
arXiv:1809.05934v2 fatcat:7sisnsbvrraytjp3bkhdlybxhq

Enhanced word clustering for hierarchical text classification

Inderjit S. Dhillon, Subramanyam Mallela, Rahul Kumar
2002 Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '02  
In this paper we propose a new information-theoretic divisive algorithm for word clustering applied to text classification.  ...  We further show that feature clustering is an effective technique for building smaller class models in hierarchical classification.  ...  We are grateful to Andrew McCallum and Byron Dom for helpful discussions. For this research, ISD was supported by a NSF CAREER Grant (No.  ... 
doi:10.1145/775047.775076 dblp:conf/kdd/DhillonMK02 fatcat:xlvbo7yp2nbzniya5zs3vqvjga

Confidence-Constrained Maximum Entropy Framework for Learning from Multi-Instance Data [article]

Behrouz Behmardi, Forrest Briggs, Xiaoli Z. Fern, Raviv Raich
2016 arXiv   pre-print
We present a maximum entropy (ME) framework for learning from multi-instance data. In this approach each bag is represented as a distribution using the principle of ME.  ...  Moreover, we compare the performance of CME with Multi-Instance Learning (MIL) state-of-the-art algorithms and show a comparable performance in terms of accuracy with reduced computational complexity.  ...  The KL divergence between two distributions obtained by the maximum entropy approach has a closed form: D(p λi p λj ) = (λ i − λ j ) T E p λ i [Φ] − (Z(λ i ) − Z(λ j )).  ... 
arXiv:1603.01901v1 fatcat:oy43ndnwgjbr3mmizrzsp4kb7m
« Previous Showing results 1 — 15 out of 4,902 results