Filters








363 Hits in 4.8 sec

The Geometric Occam's Razor Implicit in Deep Learning [article]

Benoit Dherin, Michael Munn, David G.T. Barrett
2021 arXiv   pre-print
We explore the relationship between this Geometric Occam's Razor, the Dirichlet energy and other known forms of implicit regularization.  ...  Finally, for ResNets trained on CIFAR-10, we observe that Dirichlet energy measurements are consistent with the action of this implicit Geometric Occam's Razor.  ...  The Geometric Occam's razor in 1-dimensional regression To build intuition, we begin with a simple 1-dimensional example.  ... 
arXiv:2111.15090v2 fatcat:fpbzo6al6zf67n2pyhlugvyv64

Applying Deutsch's concept of good explanations to artificial intelligence and neuroscience – an initial exploration [article]

Daniel C. Elton
2020 arXiv   pre-print
In this work we investigate Deutsch's hard-to-vary principle and how it relates to more formalized principles in deep learning such as the bias-variance trade-off and Occam's razor.  ...  We explore what role hard-to-vary explanations play in intelligence by looking at the human brain and distinguish two learning systems in the brain.  ...  Acknowledgements The author appreciates helpful discussions with Dr.  ... 
arXiv:2012.09318v1 fatcat:xcp5uto65bbpfftfvdiv4ajog4

Model averages sharpened into Occam's razors: Deep learning enhanced by Rényi entropy

David R. Bickel
2019 Zenodo  
Ensemble methods of machine learning combine neural networks or other machine learning models in order to improve predictive performance.  ...  The proposed ensemble method is based on Occam's razor idealized as adjusting hyperprior distributions over models according to a Rényi entropy of the data distribution that corresponds to each model.  ...  Occam's razor addresses that problem by adjusting the prior distribution over models for the complexity of their predictive distributions, as described in Section 2.  ... 
doi:10.5281/zenodo.3553146 fatcat:wznhvzczvndq5d2olgd3zcuptm

Model averages sharpened into Occam's razors: Deep learning enhanced by Rényi entropy

David R. Bickel
2019 Zenodo  
Ensemble methods of machine learning combine neural networks or other machine learning models in order to improve predictive performance.  ...  The proposed ensemble method is based on Occam's razor idealized as adjusting hyperprior distributions over models according to a Rényi entropy of the data distribution that corresponds to each model.  ...  Occam's razor addresses that problem by adjusting the prior distribution over models for the complexity of their predictive distributions, as described in Section 2.  ... 
doi:10.5281/zenodo.3565931 fatcat:xhqim6x6o5ckpo6saaxt4cdagy

A Formal Measure of Machine Intelligence [article]

Shane Legg, Marcus Hutter
2006 arXiv   pre-print
We believe that this measure formally captures the concept of machine intelligence in the broadest reasonable sense.  ...  In this paper we approach this problem in the following way: We take a number of well known informal definitions of human intelligence that have been given by experts, and extract their essential features  ...  Thus, although we don't usually mention Occam's razor when defining intelligence, the ability to effectively use Occam's razor is clearly a part of intelligent behaviour.  ... 
arXiv:cs/0605024v1 fatcat:c5p3ye7xf5bmvf5npqbhowxnuq

Implicit Pairs for Boosting Unpaired Image-to-Image Translation [article]

Yiftach Ginger, Dov Danon, Hadar Averbuch-Elor, Daniel Cohen-Or
2020 arXiv   pre-print
In image-to-image translation the goal is to learn a mapping from one image domain to another. In the case of supervised approaches the mapping is learned from paired samples.  ...  As a result, in recent years more attention has been given to techniques that learn the mapping from unpaired sets.  ...  Assuming a passable generator would translate a i to the neighborhood of T (a i ) by generating G A (a i ) ≈ T (a i ), and using Occam's Razor, we could expected the enriched discriminator to provide better  ... 
arXiv:1904.06913v3 fatcat:ua3ww5biubbchhuyohx4sfpsya

ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representations [article]

Samuel Pfrommer and Mathew Halm and Michael Posa
2020 arXiv   pre-print
In this work, we resolve this conflict with a smooth, implicit encoding of the structure inherent to contact-induced discontinuities.  ...  We furthermore circumvent the need to differentiate through stiff or non-smooth dynamics with a novel loss function inspired by the principles of complementarity and maximum dissipation.  ...  Occam's razor: the best parameterization (i.e. simplest explanation) is the smoothest interpolator [9] or an infinitelydifferentiable regressor [10] of the data.  ... 
arXiv:2009.11193v2 fatcat:osvc7oeaxzc2xjvvlvj2jghege

Neighborhood Mixup Experience Replay: Local Convex Interpolation for Improved Sample Efficiency in Continuous Control Tasks [article]

Ryan Sander, Wilko Schwarting, Tim Seyde, Igor Gilitschenski, Sertac Karaman, Daniela Rus
2022 arXiv   pre-print
Experience replay plays a crucial role in improving the sample efficiency of deep reinforcement learning agents.  ...  We combine our approach with recent off-policy deep reinforcement learning algorithms and evaluate on continuous control environments.  ...  The authors thank the MIT SuperCloud and Lincoln Laboratory Supercomputing Center (Reuther et al. 2018) for providing HPC and consultation resources that have contributed to the research results reported  ... 
arXiv:2205.09117v1 fatcat:o5bvbw77u5dbjnhbzqre4joo5i

Universal Intelligence: A Definition of Machine Intelligence

Shane Legg, Marcus Hutter
2007 Minds and Machines  
We then show how this formal definition is related to the theory of universal optimal learning agents.  ...  We believe that this equation formally captures the concept of machine intelligence in the broadest reasonable sense.  ...  Acknowledgements This work was supported by the Swiss NSF grant 200020-107616.  ... 
doi:10.1007/s11023-007-9079-x fatcat:bugwjjrkkzbfre2rxdx7dkvd3i

Universal Intelligence: A Definition of Machine Intelligence [article]

Shane Legg, Marcus Hutter
2007 arXiv   pre-print
We then show how this formal definition is related to the theory of universal optimal learning agents.  ...  We believe that this equation formally captures the concept of machine intelligence in the broadest reasonable sense.  ...  Acknowledgements This work was supported by the Swiss NSF grant 200020-107616.  ... 
arXiv:0712.3329v1 fatcat:2xpyeksxmbeyrlun34bmrc444q

BAE-NET: Branched Autoencoder for Shape Co-Segmentation [article]

Zhiqin Chen, Kangxue Yin, Matthew Fisher, Siddhartha Chaudhuri, Hao Zhang
2019 arXiv   pre-print
By complementing the shape reconstruction loss with a label loss, BAE-NET is easily tuned for one-shot learning.  ...  Importantly, the decoder is branched: each branch learns a compact representation for one commonly recurring part of the shape collection, e.g., airplane wings.  ...  If one were to abide by Occam's razor, then the best explanation would be the simplest one.  ... 
arXiv:1903.11228v2 fatcat:htxwsw6wovdjzmzrb2hiveb6zi

Few-Shot Bayesian Imitation Learning with Logical Program Policies

Tom Silver, Kelsey R. Allen, Alex K. Lew, Leslie Pack Kaelbling, Josh Tenenbaum
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
Humans can learn many novel tasks from a very small number (1–5) of demonstrations, in stark contrast to the data requirements of nearly tabula rasa deep learning methods.  ...  In experiments, we study six strategy games played on a 2D grid with one shared DSL.  ...  Acknowledgements We acknowledge support from NSF grants 1523767 and 1723381; from ONR grant N00014-13-1-0333; from AFOSR grant FA9550-17-1-0165; from ONR grant 10257 N00014-18-1-2847; from Honda Research; and from the  ... 
doi:10.1609/aaai.v34i06.6587 fatcat:2iz4sfzmevfmxcodfqs7c2yuwu

Universal Artificial Intelligence [chapter]

Tom Everitt, Marcus Hutter
2018 Foundations of Trusted Autonomy  
Since the inception of the AI research field in the mid-twentieth century, a range of practical and theoretical approaches have been investigated.  ...  As most research studies focus on one narrow question, it is essential that the value of each isolated result can be appreciated in light of a broader framework or goal formulation.  ...  To avoid overfitting, smaller MDPs are also preferred, in line with Occam's razor. The computational flow of a MDP agent is depicted in Fig. 2 .5.  ... 
doi:10.1007/978-3-319-64816-3_2 fatcat:pvbspss75bcftktbrhbyjozyom

Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes [article]

Lei Wu, Zhanxing Zhu, Weinan E
2017 arXiv   pre-print
It is widely observed that deep learning models with learned parameters generalize well, even with much more model parameters than the number of training samples.  ...  We systematically investigate the underlying reasons why deep neural networks often generalize well, and reveal the difference between the minima (with the same training error) that generalize well and  ...  This intuitive explanation is called Occam's razor, and No Free Lunch theorem [20] and also related to the minimum description length (MDL) theory [18, 15] .  ... 
arXiv:1706.10239v2 fatcat:jfshieu6urfjnjmhh6pkw3tmru

DiGS : Divergence guided shape implicit neural representation for unoriented point clouds [article]

Yizhak Ben-Shabat, Chamin Hewa Koneputugodage, Stephen Gould
2022 arXiv   pre-print
Existing INRs require point coordinates to learn the implicit level sets of the shape.  ...  Shape implicit neural representations (INRs) have recently shown to be effective in shape analysis and reconstruction tasks.  ...  This can be viewed as an Occam's Razor principle, where simpler solutions are often better explanations/predictions.  ... 
arXiv:2106.10811v2 fatcat:fqak6xeid5amrfeh6qtfwbvq5q
« Previous Showing results 1 — 15 out of 363 results