A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Hyperopt-Sklearn
[chapter]
2019
Automated Machine Learning
Preserved Fulltext
Hyperopt-sklearn is a software project that provides automated algorithm configuration of the Scikit-learn machine learning library. Following Auto-Weka, we take the view that the choice of classifier and even the choice of preprocessing module can be taken together to represent a single large hyperparameter optimization problem. We use Hyperopt to define a search space that encompasses many standard components (e.g. SVM, RF, KNN, PCA, TFIDF) and common patterns of composing them together. We
doi:10.1007/978-3-030-05318-5_5
fatcat:j7mxmmo7lzhqtn5uardxjgitzu
more »
... monstrate, using search algorithms in Hyperopt and standard benchmarking data sets (MNIST, 20-Newsgroups, Convex Shapes), that searching this space is practical and effective. In particular, we improve on best-known scores for the model space for both MNIST and Convex Shapes at the time of release. Introduction Relative to deep networks, algorithms such as Support Vector Machines (SVMs) and Random Forests (RFs) have a small-enough number of hyperparameters that manual tuning and grid or random search provides satisfactory results. Taking a step back though, there is often no particular reason to use either an SVM or an RF when they are both computationally viable. A model-agnostic practitioner may simply prefer to go with the one that provides greater accuracy. In this light, the choice of classifier can be seen as hyperparameter alongside the C-value in the SVM and the max-treedepth of the RF. Indeed the choice and configuration of preprocessing components may likewise be seen as part of the model selection/hyperparameter optimization problem.
Predicting Genre Labels For Artist Using Freedb
2006
Zenodo
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
[TODO] Add abstract here.
doi:10.5281/zenodo.1416447
fatcat:dhnqeh7f5jetblhigxezc6mcwq
Active Perception and Representation for Robotic Manipulation
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The vast majority of visual animals actively control their eyes, heads, and/or bodies to direct their gaze toward different parts of their environment. In contrast, recent applications of reinforcement learning in robotic manipulation employ cameras as passive sensors. These are carefully placed to view a scene from a fixed pose. Active perception allows animals to gather the most relevant information about the world and focus their computational resources where needed. It also enables them to
arXiv:2003.06734v1
fatcat:c7itiu3d7zgzvgdyqoer23fzfi
more »
... iew objects from different distances and viewpoints, providing a rich visual experience from which to learn abstract representations of the environment. Inspired by the primate visual-motor system, we present a framework that leverages the benefits of active perception to accomplish manipulation tasks. Our agent uses viewpoint changes to localize objects, to learn state representations in a self-supervised manner, and to perform goal-directed actions. We apply our model to a simulated grasping task with a 6-DoF action space. Compared to its passive, fixed-camera counterpart, the active model achieves 8% better performance in targeted grasping. Compared to vanilla deep Q-learning algorithms, our model is at least four times more sample-efficient, highlighting the benefits of both active perception and representation learning.
Scalable Genre And Tag Prediction With Spectral Covariance
2010
Zenodo
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
[TODO] Add abstract here.
doi:10.5281/zenodo.1416942
fatcat:gv4b7kr6ffexraz4nxip7fz7qi
Review of Suppes 1957 Proposals For Division by Zero
2021
Transmathematica
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
We review the exposition of division by zero and the definition of total arithmetical functions in "Introduction to Logic" by Patrick Suppes, 1957, and provide a hyperlink to the archived text. This book is a pedagogical introduction to first-order predicate calculus with logical, mathematical, physical and philosophical examples, some presented in exercises. It is notable for (i) presenting division by zero as a problem worthy of contemplation, (ii) considering five totalisations of real
doi:10.36285/tm.53
fatcat:qyepiya3vfenth5cylkoiyaqsy
more »
... etic, and (iii) making the observation that each of these solutions to "the problem of division by zero" has both advantages and disadvantages -- none of the proposals being fully satisfactory. We classify totalisations by the number of non-real symbols they introduce, called their Extension Type. We compare Suppes' proposals for division by zero to more recent proposals. We find that all totalisations of Extension Type 0 are arbitrary, hence all non-arbitrary totalisations are of Extension Type at least 1. Totalisations of the differential and integral calculus have Extension Type at least 2. In particular, Meadows have Extension Type 1, Wheels have Extension Type 2, and Transreal numbers have Extension Type 3. It appears that Suppes was the modern originator of the idea that all real numbers divided by zero are equal to zero. This has Extension Type 0 and is, therefore, arbitrary.
Autoregressive Policies for Continuous Control Deep Reinforcement Learning
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Reinforcement learning algorithms rely on exploration to discover new behaviors, which is typically achieved by following a stochastic policy. In continuous control tasks, policies with a Gaussian distribution have been widely adopted. Gaussian exploration however does not result in smooth trajectories that generally correspond to safe and rewarding behaviors in practical tasks. In addition, Gaussian policies do not result in an effective exploration of an environment and become increasingly
arXiv:1903.11524v1
fatcat:scmpbz253jfcbkadiwz2ryizsa
more »
... fficient as the action rate increases. This contributes to a low sample efficiency often observed in learning continuous control tasks. We introduce a family of stationary autoregressive (AR) stochastic processes to facilitate exploration in continuous control domains. We show that proposed processes possess two desirable features: subsequent process observations are temporally coherent with continuously adjustable degree of coherence, and the process stationary distribution is standard normal. We derive an autoregressive policy (ARP) that implements such processes maintaining the standard agent-environment interface. We show how ARPs can be easily used with the existing off-the-shelf learning algorithms. Empirically we demonstrate that using ARPs results in improved exploration and sample efficiency in both simulated and real world domains, and, furthermore, provides smooth exploration trajectories that enable safe operation of robotic hardware.
Benchmarking Reinforcement Learning Algorithms on Real-World Robots
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
To analyze the hyper-parameter sensitivity within tasks and consistency across tasks, we perform a random search (Bergstra & Bengio 2012) of seven hyper-parameters of each algorithm on UR-Reacher-2 and ...
arXiv:1809.07731v1
fatcat:ticoo72gvnb5jfbnm3oiu4jdd4
Algorithms for Hyper-Parameter Optimization
2011
Neural Information Processing Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Several recent advances to the state of the art in image classification benchmarks have come from better configurations of existing techniques rather than novel approaches to feature learning. Traditionally, hyper-parameter optimization has been the job of humans because they can be very efficient in regimes where only a few trials are possible. Presently, computer clusters and GPU processors make it possible to run more trials and we show that algorithmic approaches can find better results. We
dblp:conf/nips/BergstraBBK11
fatcat:jwye6abnnfeeneqh3owfugflgq
more »
... present hyper-parameter optimization results on tasks of training neural networks and deep belief networks (DBNs). We optimize hyper-parameters using random search and two new greedy sequential methods based on the expected improvement criterion. Random search has been shown to be sufficiently efficient for learning neural networks for several datasets, but we show it is unreliable for training DBNs. The sequential algorithms are applied to the most difficult DBN learning problems from [1] and find significantly better results than the best previously reported. This work contributes novel techniques for making response surface models P (y|x) in which many elements of hyper-parameter assignment (x) are known to be irrelevant given particular values of other elements.
Random Search for Hyper-Parameter Optimization
2012
Journal of machine learning research
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Acknowledgments This work was supported by the National Science and Engineering Research Council of Canada and Compute Canada, and implemented with Theano (Bergstra et al., 2010) . ...
dblp:journals/jmlr/BergstraB12
fatcat:p2ekjdmib5cf3aebxzmn5f3ane
Theano: new features and speed improvements
[article]
2012
arXiv
pre-print
Preserved Fulltext
The Internet Archive has a preservation copy of this work in our general collections.
The file type is application/pdf.
Bergstra et al. (2010 Bergstra et al. ( , 2011 , as well as Theano's website 1 have more in-depth descriptions and examples. ...
Bergstra et al. (2010) showed that Theano was faster than many other tools available at the time, including Torch5. ...
arXiv:1211.5590v1
fatcat:ep2hj6pufffvtevzl7m3yvhtua
Pylearn2: a machine learning research library
[article]
2013
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
This idea grew out of James Bergstra's Theano-linear module which has since been incorporated into Pylearn2. ...
arXiv:1308.4214v1
fatcat:nwsear5oenhullzswvv5p6yzae
Slow, Decorrelated Features for Pretraining Complex Cell-like Networks
2009
Neural Information Processing Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
We introduce a new type of neural network activation function based on recent physiological rate models for complex cells in visual area V1. A single-hiddenlayer neural network of this kind of model achieves 1.50% error on MNIST. We also introduce an existing criterion for learning slow, decorrelated features as a pretraining strategy for image models. This pretraining strategy results in orientation-selective features, similar to the receptive fields of complex cells. With this pretraining,
dblp:conf/nips/BergstraB09
fatcat:7fsdv7zsyjhsxn3atbrre36vgq
more »
... same single-hidden-layer model achieves 1.34% error, even though the pretraining sample distribution is very different from the fine-tuning distribution. To implement this pretraining strategy, we derive a fast algorithm for online learning of decorrelated features such that each iteration of the algorithm runs in linear time with respect to the number of features.
Setting up a Reinforcement Learning Task with a Real-World Robot
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Reinforcement learning is a promising approach to developing hard-to-engineer adaptive solutions for complex and diverse robotic tasks. However, learning with real-world robots is often unreliable and difficult, which resulted in their low adoption in reinforcement learning research. This difficulty is worsened by the lack of guidelines for setting up learning tasks with robots. In this work, we develop a learning task with a UR5 robotic arm to bring to light some key elements of a task setup
arXiv:1803.07067v1
fatcat:uxv2tmvg2zda3duyqysz4izp6u
more »
... d study their contributions to the challenges with robots. We find that learning performance can be highly sensitive to the setup, and thus oversights and omissions in setup details can make effective learning, reproducibility, and fair comparison hard. Our study suggests some mitigating steps to help future experimenters avoid difficulties and pitfalls. We show that highly reliable and repeatable experiments can be performed in our setup, indicating the possibility of reinforcement learning research extensively based on real-world robots.
Aggregate features and ADABOOST for music classification
2006
Machine Learning
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2007; you can also visit the original URL.
The file type is application/pdf.
We participated in two contests: Audio Genre Classification Bergstra et al. (2005a) , and Audio Artist Identification Bergstra et al. (2005b) . ...
doi:10.1007/s10994-006-9019-7
fatcat:dau2hrwiqbbopkif47muewqf5i
A Spike and Slab Restricted Boltzmann Machine
2011
Journal of machine learning research
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
We introduce the spike and slab Restricted Boltzmann Machine, characterized by having both a real-valued vector, the slab, and a binary variable, the spike, associated with each unit in the hidden layer. The model possesses some practical properties such as being amenable to Block Gibbs sampling as well as being capable of generating similar latent representations of the data to the recently introduced mean and covariance Restricted Boltzmann Machine. We illustrate how the spike and slab
dblp:journals/jmlr/CourvilleBB11
fatcat:yo6hkv2qynfi5cthrgz3kx7whq
more »
... ted Boltzmann Machine achieves competitive performance on the CIFAR-10 object recognition task.
« Previous
Showing results 1 — 15 out of 470 results