12,425 Hits in 8.6 sec

Logistic Regression: The Importance of Being Improper [article]

Dylan J. Foster, Satyen Kale, Haipeng Luo, Mehryar Mohri, Karthik Sridharan
2018 arXiv   pre-print
Starting with the simple observation that the logistic loss is $1$-mixable, we design a new efficient improper learning algorithm for online logistic regression that circumvents the aforementioned lower  ...  Finally, we give information-theoretic bounds on the optimal rates for improper logistic regression with general function classes, thereby characterizing the extent to which our improvement for linear  ...  DF thanks Matus Telgarsky for sparking an interest in logistic regression through a series of talks at the Simons Institute.  ... 
arXiv:1803.09349v2 fatcat:krryhyeisngzzew2q63fb6iwhm

Bayesian Multivariate Logistic Regression

Sean M. O'Brien, David B. Dunson
2004 Biometrics  
Motivated by these problems, we propose a new type of multivariate logistic distribution that can be used to construct a likelihood for multivariate logistic regression analysis of binary and categorical  ...  Bayesian analyses of multivariate binary or categorical outcomes typically rely on probit or mixed effects logistic regression models which do not have a marginal logistic structure for the individual  ...  A major drawback of the Bayesian approach to mixed effects logistic regression is poor performance when diffuse or improper priors are chosen.  ... 
doi:10.1111/j.0006-341x.2004.00224.x pmid:15339297 fatcat:qhi5ne5ei5bpfc4wkqfzlwdi2y

Efficient improper learning for online logistic regression [article]

Rémi Jézéquel
2020 arXiv   pre-print
We consider the setting of online logistic regression and consider the regret with respect to the 2-ball of radius B.  ...  ., 2018] showed that the lower bound does not apply to improper algorithms and proposed a strategy based on exponential weights with prohibitive computational complexity.  ...  Acknowledgements This work was funded in part by the French government under management of Agence Nationale de la Recherche as part of the "Investissements d'avenir" program, reference ANR-19-P3IA-0001  ... 
arXiv:2003.08109v3 fatcat:qormkpmwivgglirfwgble2uieq

Polytomous Logistic Regression: Methods and Application [chapter]

J. Engel
1989 Statistical Modelling  
A justification towards the application of Polytomous Logistic Regression is made evident in this study.  ...  The purpose of this study was to employ Polytomous Logistic Regression to predict inhabitants' modes of waste disposal practices by examining several indicator variables.  ...  More literature on Polytomous Logistic Regression can be found in [25] [26] .  ... 
doi:10.1007/978-1-4612-3680-1_15 fatcat:2fouguncsffjxpn4znfrz2rh6i

Simulation-based Regularized Logistic Regression

Robert B. Gramacy, Nicholas G. Polson
2012 Bayesian Analysis  
We develop an omnibus framework for regularized logistic regression by simulationbased inference, exploiting two important results on scale mixtures of normals.  ...  By carefully choosing a hierarchical model for the likelihood by one type of mixture, and how regularization may be implemented by another, we obtain subtly different MCMC schemes with varying efficiency  ...  The authors would like thank Matt Taddy for interesting discussions on the efficient handling of Binomial data, extensions to Multinomial regression, and EM code for the MAP estimator(s).  ... 
doi:10.1214/12-ba719 fatcat:gyxej4edb5gmhf7tdco3liwueq

Efficient Methods for Online Multiclass Logistic Regression [article]

Naman Agarwal, Satyen Kale, Julian Zimmert
2021 arXiv   pre-print
., 2018) has highlighted the importance of improper predictors for achieving "fast rates" in the online multiclass logistic regression problem without suffering exponentially from secondary problem parameters  ...  This yields the first practical algorithm for online multiclass logistic regression, resolving an open problem of Foster et al.(2018).  ...  Logistic regres- sion: The importance of being improper. In Conference On Learning Theory, pages 167-208. PMLR, 2018. Yoav Freund and Robert E Schapire.  ... 
arXiv:2110.03020v2 fatcat:cx3pfnihyvc5losso633fi4ude

Simulation-based Regularized Logistic Regression [article]

Robert B. Gramacy, Nicholas G. Polson
2012 arXiv   pre-print
In this paper, we develop a simulation-based framework for regularized logistic regression, exploiting two novel results for scale mixtures of normals.  ...  By carefully choosing a hierarchical model for the likelihood by one type of mixture, and implementing regularization with another, we obtain new MCMC schemes with varying efficiency depending on the data  ...  The authors would like thank Matt Taddy for interesting discussions on the efficient handling of Binomial data, extensions to Multinomial regression, and EM code for the MAP estimator(s).  ... 
arXiv:1005.3430v4 fatcat:d525karalfcrhfioylsiafi5ry

Logistic Regression for Prospectivity Modeling [chapter]

Samuel Kost, Oliver Rheinbach, Helmut Schaeben
2020 Lecture Notes in Computational Science and Engineering  
Linear Regression . Logistic Regression . Logistic Regression . Logistic Regression  ...  Statistical methods such as logistic regression underestimate the probability of rare events and tend to be biased towards the majority class which is the less important class.  ... 
doi:10.1007/978-3-030-55874-1_81 fatcat:pwr2yt6lzbenvm56ubyexayaqu

Variable selection for multivariate logistic regression models

Ming-Hui Chen, Dipak K. Dey
2003 Journal of Statistical Planning and Inference  
In this paper, we use multivariate logistic regression models to incorporate correlation among binary response data.  ...  The propriety of the proposed informative prior is investigated in detail.  ...  D'Amico of the Joint Center for Radiation Therapy at Harvard Medical School for providing the prostate cancer data sets. Dr. Chen's research was partially supported by NSF grant No.  ... 
doi:10.1016/s0378-3758(02)00284-7 fatcat:t3u2rv6of5dkpbqaqnetopp4jy

Logistic regression with weight grouping priors

M. Korzeń, S. Jaroszewicz, P. Klęsk
2013 Computational Statistics & Data Analysis  
A generalization of the commonly used Maximum Likelihood based learning algorithm for the logistic regression model is considered.  ...  It is well known that using the Laplace prior (L 1 penalty) on model coefficients leads to a variable selection effect, when most of the coefficients vanish.  ...  Acknowledgements This work has been partly financed by the Polish Ministry of Science and Higher Education from research funds for the years 2010-2012. Research project no.: N N516 424938.  ... 
doi:10.1016/j.csda.2013.03.013 fatcat:ehr4jbglnngbhh4yaf6qsnxq5y

Dealing with Separation in Logistic Regression Models

Carlisle Rainey
2016 Political Analysis  
While Jeffreys' prior has the advantage of being automatic, I show that it often provides too much prior information, producing smaller point estimates and narrower confidence intervals than even highly  ...  distribution of quantities of interest, estimate the subsequent model, and summarize the results.  ...  , but uses generalized estimating equations, which we might interpret as having an improper, uniform prior on the logistic regression coefficients from minus infinity to plus infinity.  ... 
doi:10.1093/pan/mpw014 fatcat:jo7c6g43tvgw3nlfwpfv7iamma

An improper estimator with optimal excess risk in misspecified density estimation and logistic regression [article]

Jaouad Mourtada, Stéphane Gaïffas
2021 arXiv   pre-print
For logistic regression, SMP provides a non-Bayesian approach to calibration of probabilistic predictions relying on virtual samples, and can be computed by solving two logistic regressions.  ...  Being an improper (out-of-model) procedure, SMP improves over within-model estimators such as the maximum likelihood estimator, whose excess risk degrades under misspecification.  ...  the distribution P , and can typically be arbitrarily large, as will be seen below in the case of logistic regression.  ... 
arXiv:1912.10784v3 fatcat:m6hmy7hboncxpjfeklwqeyeoam

Mixability made efficient: Fast online multiclass logistic regression [article]

Rémi Jézéquel
2021 arXiv   pre-print
For example, in the case of multiclass logistic regression, the aggregating forecaster (Foster et al. (2018)) achieves a regret of $O(\log(Bn))$ whereas Online Newton Step achieves $O(e^B\log(n))$ obtaining  ...  a double exponential gain in $B$ (a bound on the norm of comparative functions).  ...  For binary logistic regression, two efficient improper algorithms have been proposed in the literature.  ... 
arXiv:2110.03960v1 fatcat:a5mdaq5ww5hazmqxqqfkhvffgm

Geometric ergodicity of Polya-Gamma Gibbs sampler for Bayesian logistic regression with a flat prior [article]

Xin Wang, Vivekananda Roy
2018 arXiv   pre-print
In the absence of any prior information, an improper flat prior is often used for the regression coefficients in Bayesian logistic regression models.  ...  The logistic regression model is the most popular model for analyzing binary data.  ...  Let β ∈ R p be the unknown vector of regression coefficients.  ... 
arXiv:1802.06248v3 fatcat:xdxy76pt55gj5f5a3avklv6cty

Mind reading with regularized multinomial logistic regression

Heikki Huttunen, Tapio Manninen, Jukka-Pekka Kauppi, Jussi Tohka
2012 Machine Vision and Applications  
The method is based on a regularized logistic regression model, whose ecient feature selection is critical for cases with more measurements than samples.  ...  In this paper, we consider the problem of multinomial classication of magnetoencephalography (MEG) data.  ...  Instead of the symmetric logistic regression model in Equation 2, we use a traditional logistic regression model: p(x) = 1 1 + exp(β 0 + β T x) , (4) where p(x) estimates the probability of class 1 given  ... 
doi:10.1007/s00138-012-0464-y fatcat:5u2dlz2s7neevlo35g3vadzdiy
« Previous Showing results 1 — 15 out of 12,425 results