Filters








8,645 Hits in 6.1 sec

Information Bottlenecks, Causal States, and Statistical Relevance Bases: How to Represent Relevant Information in Memoryless Transduction [article]

Cosma Rohilla Shalizi, James P. Crutchfield
2000 arXiv   pre-print
This brief note explains the connections between three approaches to this problem: the recently introduced information-bottleneck method, the computational mechanics approach to inferring optimal models  ...  , and Salmon's statistical relevance basis.  ...  ACKNOWLEDGMENTS This work was partially supported by the SFI Computation, Dynamics, and Learning Program, by AFOSR via NSF grant PHY-9970158, and by DARPA under contract F30602-00-2-0583.  ... 
arXiv:nlin/0006025v1 fatcat:5dbqa52gbjbafi5mw6y2bw6p34

Understanding Deep Learning with Statistical Relevance

Tim Räz
2022 Philosophy of Science  
This establishes a link to deep neural networks via the so-called Information Bottleneck method, an information-theoretic framework, according to which deep neural networks implicitly solve an optimization  ...  It is proved that homogeneous partitions, the core notion of Salmon's model, are equivalent to minimal sufficient statistics, an important notion from statistical inference.  ...  philosophy of science colloquium in the fall of 2019 in Bern for comments, Claus Beisbart and Michael Vock for valuable feedback on earlier versions of the paper, Samuel Portmann for discussions and help with  ... 
doi:10.1017/psa.2021.12 fatcat:4sh26qc5lzeu3ipish7tao376m

INFORMATION BOTTLENECKS, CAUSAL STATES, AND STATISTICAL RELEVANCE BASES: HOW TO REPRESENT RELEVANT INFORMATION IN MEMORYLESS TRANSDUCTION

COSMA ROHILLA SHALIZI, JAMES P. CRUTCHFIELD
2002 Advances in Complex Systems  
This brief note explains the connections between three approaches to this problem: the recently introduced information-bottleneck method, the computational mechanics approach to inferring optimal models  ...  , and Salmon's statistical relevance basis.  ...  ACKNOWLEDGMENTS This work was partially supported by the SFI Computation, Dynamics, and Learning Program, by AFOSR via NSF grant PHY-9970158, and by DARPA under contract F30602-00-2-0583.  ... 
doi:10.1142/s0219525902000481 fatcat:i7bqn2sqqnafvmvdt4pv2sx3ny

From Statistical to Causal Learning [article]

Bernhard Schölkopf, Julius von Kügelgen
2022 arXiv   pre-print
We describe basic ideas underlying research to build and understand artificially intelligent systems: from symbolic approaches via statistical learning to interventional models relying on concepts of causality  ...  Some of the hard open problems of machine learning and AI are intrinsically related to causality, and progress may require advances in our understanding of how to model and infer causality from data.  ...  Ackowledgements Many thanks to all past and present members of the Tübingen causality team, and to Cian Eastwood and Elias Bareinboim for feedback on the manuscript.  ... 
arXiv:2204.00607v1 fatcat:u7ylajln7rgzfgz7cq2thc5cmy

Quantum Statistical Inference [article]

Zhikuan Zhao
2018 arXiv   pre-print
Secondly, I look into the notion of quantum causality and apply it to inferring spatial and temporal quantum correlations, and present an analytical toolkit for causal inference in quantum data.  ...  I then apply the quantum enhanced GPs to Bayesian deep learning and present an experimental demonstration on contemporary hardware and simulators.  ...  In modern statistical inference and machine learning applications, matrix inversion presents a computational bottleneck when the dimensionality n of the underlying problem grows.  ... 
arXiv:1812.04877v1 fatcat:ne6bq6pdorg6lo4pa77j4omoke

Predictive Rate-Distortion for Infinite-Order Markov Processes

Sarah E. Marzen, James P. Crutchfield
2016 Journal of statistical physics  
exponentially with length.  ...  Predictive rate-distortion analysis suffers from the curse of dimensionality: clustering arbitrarily long pasts to retain information about arbitrarily long futures requires resources that typically grow  ...  [30] is an information bottleneck approach to predictive inference that does not take the form of Eq. (7).  ... 
doi:10.1007/s10955-016-1520-1 fatcat:r7ohn2zs2jcbxj6l2sdamnfcmi

Statistical Model Fitting and Model Selection in Pedestrian Dynamics Research

Nikolai W.F. Bode, Enrico Ronchi
2019 Collective Dynamics  
Statistical model fitting and model selection are a suitable approach to this problem and here we review the concepts and literature related to this methodology in the context of pedestrian dynamics.  ...  The survey of the literature we present provides many methodological starting points and we suggest that the particular challenges to statistical modelling in pedestrian dynamics make this an inherently  ...  An important first step is for all researchers to openly engage with and report on the limitations of their statistical modelling approaches.  ... 
doi:10.17815/cd.2019.20 fatcat:5aed7zhc6vgvnd3qy5zpz6lqna

Statistical Schema Induction [chapter]

Johanna Völker, Mathias Niepert
2011 Lecture Notes in Computer Science  
In this paper, we present a statistical approach to the induction of expressive schemas from large RDF repositories.  ...  We describe in detail the implementation of this approach and report on an evaluation that we conducted using several data sets including DBpedia.  ...  Statistical Schema Induction In the following, we describe in detail our approach to inducing or enriching the schema of an RDF repository through its SPARQL endpoint.  ... 
doi:10.1007/978-3-642-21034-1_9 fatcat:sx2atiybcvbv5mvejlqd3af3mq

Efficient Statistical Performance Modeling for Autonomic, Service-Oriented Systems

Rui Zhang, Alan Bivens, Iead Rezek
2007 2007 IEEE International Parallel and Distributed Processing Symposium  
Recent work addressing this issue frequently used statistically learned models which were derived entirely from data.  ...  Simulations and applications in actual environments show significant reductions in learning time, better accuracy and stronger tolerance to small learning data sets.  ...  These statistical learning approaches do not assume human involvement and require little domain analysis.  ... 
doi:10.1109/ipdps.2007.370251 dblp:conf/ipps/ZhangBR07 fatcat:c2qcur5rvvgptc5omwb7umhbxu

Social networks and statistical relational learning: a survey

Floriana Esposito, Stefano Ferilli, Teresa M.A. Basile, Nicola Di Mauro
2012 International Journal of Social Network Mining  
Statistical relational learning (SRL) is a very promising approach to SNM, since it combines expressive representation formalisms, able to model complex relational networks, with statistical methods able  ...  Social networks potentially represent an invaluable source of information that can be exploited for scientific and commercial purposes.  ...  In order to have sufficiently robust and efficient systems able to deal with large quantities of possibly noisy data a lot of work, known under the names of statistical relational learning (SRL) (Getoor  ... 
doi:10.1504/ijsnm.2012.051057 fatcat:6i7n2zrrj5geho3eh6nicay5nm

Topographic Product Models Applied to Natural Scene Statistics

Simon Osindero, Max Welling, Geoffrey E. Hinton
2006 Neural Computation  
We present an energy-based model that uses a product of generalised Student-t distributions to capture the statistical structure in datasets.  ...  Using patches of natural scenes we demonstrate that our approach represents a viable alternative to "independent components analysis" as an interpretive model of biological visual systems.  ...  from a causal approach.  ... 
doi:10.1162/089976606775093936 pmid:16378519 fatcat:qei4wvbbwze37kcf54wnei54my

Three Research Challenges at the Intersection of Machine Learning, Statistical Induction, and Systems

Moisés Goldszmidt, Ira Cohen, Armando Fox, Steve Zhang
2005 USENIX Workshop on Hot Topics in Operating Systems  
additional information.  ...  Recent research activity [2, 12, 27, 10, 1] has shown encouraging results for performance debugging and failure diagnosis and detection in systems by using approaches based on automatically inducing models  ...  How do we know that the training data is sufficiently representative of the data seen during production operations-an implicit assumption of most of these approaches?  ... 
dblp:conf/hotos/GoldszmidtCFZ05 fatcat:y6m5zezuabh6vkjzpwnnsd64iy

Machine Learning, Neural, and Statistical Classification

John F. Elder IV, Donald Michie, David J. Spiegelhalter, Charles C. Taylor
1996 Journal of the American Statistical Association  
This is not the case with some statistical and neural learning algorithms.  ...  Traditional and statistical approaches An empirical study of a point awarding approach to Credit Scoring is made by Häussler (1979 Häussler ( , 1981a Häussler ( , 1981b .  ...  In all the approaches described, there was an aspiration to generate comprehensible control rules, sometimes at the cost of an additional learning stage.  ... 
doi:10.2307/2291432 fatcat:mg6mr2lvjnczphrzq4t3iqjoay

ACE: A Novel Approach for the Statistical Analysis of Pairwise Connectivity [article]

Krempl, Georg and Kottke, Daniel and Pham Minh, Tuan
2021 arXiv   pre-print
This paper proposes a faster and more flexible approach for analysing such delayed correlated activity: a statistical approach for the Analysis of Connectivity in spiking Events (ACE), based on the idea  ...  Analysing correlations between streams of events is an important problem.  ...  Related work A further group of model-free approaches are based on information theoretic approaches.  ... 
arXiv:2108.04289v1 fatcat:bp5hxi7zsfeindlkc2lggpbpoe

Analysing Shortcomings of Statistical Parametric Speech Synthesis [article]

Gustav Eje Henter, Simon King, Thomas Merritt, Gilles Degottex
2018 arXiv   pre-print
The methodology is accompanied by an example that carefully measures and compares the severity of perceptual limitations imposed by vocoding as well as other factors such as the statistical model and its  ...  In particular, we discuss issues with vocoding and present a general methodology for quantifying the effect of any of the many assumptions and design choices that hold SPSS back.  ...  Surprisingly, rich linguistic context alone may never be sufficient for achieving truly convincing speech output, even with utterance-length contextual information, and a model close to true speech.  ... 
arXiv:1807.10941v1 fatcat:mlc6jcik2bhqrog5bc5yqttxry
« Previous Showing results 1 — 15 out of 8,645 results