15 Hits in 6.8 sec

Batch Normalization Provably Avoids Rank Collapse for Randomly Initialised Deep Networks [article]

Hadi Daneshmand, Jonas Kohler, Francis Bach, Thomas Hofmann, Aurelien Lucchi
2020 arXiv   pre-print
In this work we highlight the fact that batch normalization is an effective strategy to avoid rank collapse for both linear and ReLU networks.  ...  Randomly initialized neural networks are known to become harder to train with increasing depth, unless architectural enhancements like residual connections and batch normalization are used.  ...  Batch normalization provably prevents rank collapse Since our empirical observations hold equally for both non-linear and linear networks, we here focus on improving the theoretical understanding in the  ... 
arXiv:2003.01652v3 fatcat:sifsf7zdffahnkkfl7w6bf5uyi

Identifying and Exploiting Structures for Reliable Deep Learning [article]

Amartya Sanyal
2021 arXiv   pre-print
To do this, we identify structures in deep neural networks that can be exploited to mitigate the above causes of unreliability of deep learning algorithms.  ...  The extraordinary performance of these systems often gives the impression that they can be used to revolutionise our lives for the better.  ...  Random-GAN is a randomly initialised GAN with the same architecture as SRN-GAN.  ... 
arXiv:2108.07083v1 fatcat:lducrn5tlfeqvpxevz6gukfvse

Building Compact and Robust Deep Neural Networks with Toeplitz Matrices [article]

Alexandre Araujo
2021 arXiv   pre-print
Deep neural networks are state-of-the-art in a wide variety of tasks, however, they exhibit important limitations which hinder their use and deployment in real-world applications.  ...  When developing and training neural networks, the accuracy should not be the only concern, neural networks must also be cost-effective and reliable.  ...  We train our networks for 200 epochs with a batch size of 200.  ... 
arXiv:2109.00959v1 fatcat:75yhvilerre47leutzanaul67m

Open-Ended Learning Leads to Generally Capable Agents [article]

Open Ended Learning Team, Adam Stooke, Anuj Mahajan, Catarina Barros, Charlie Deck, Jakob Bauer, Jakub Sygnowski, Maja Trebacz, Max Jaderberg, Michael Mathieu, Nat McAleese, Nathalie Bradley-Schmieg (+6 others)
2021 arXiv   pre-print
and Tom Hudson for additional environment art and support.  ...  Acknowledgements We would like to thank Simon Osindero, Guy Lever, and Oriol Vinyals for reviewing the manuscript, Satinder Singh and Koray Kavukcuoglu for support of the project, and Marcus Wainwright  ...  Racanière et al. (2020) perform a curriculum over environment goals in randomly initialised 2D and 3D worlds. A setter generates goals for a solver agent.  ... 
arXiv:2107.12808v2 fatcat:wp5lbeezmrb6rdsbqf6etgurtq

Fake BTS Attacks of GSM System on Software Radio Platform

Yubo Song, Kan Zhou, Xi Chen
2012 Journal of Networks  
On the other hand, the keycontrolled puncturing mechanism deletes the parity bits randomly, which ensures a high error correction capability for the Turbo code.  ...  The first paper, "Extensive Design for Attack's Recognition and Resistance of Survivable Network" by Hui Xu and Xiang Gu, gives a solution of attack defense of survivable network based on extentics.  ...  Unified trust values or a simple trust rank is insufficient for making privacy disclose decision.  ... 
doi:10.4304/jnw.7.2.275-281 fatcat:3ejjimllcvewjeifopbvhl3zmi

Sample efficiency, transfer learning and interpretability for deep reinforcement learning

Kailash Arulkumaran, Anil Bharath
Deep learning has revolutionised artificial intelligence, where the application of increased compute to train neural networks on large datasets has resulted in improvements in real-world applications such  ...  Deep reinforcement learning (DRL) has similarly shown impressive results in board and video games, but less so in real-world applications such as robotic control.  ...  Classi- fying Options for Deep Reinforcement Learning. IJCAI Deep Reinforcement Learning Workshop. Analysing Deep Reinforcement Learning Agents Trained with Domain Randomisation.  ... 
doi:10.25560/80978 fatcat:tc4ylokvhzes7muukkvuikmngy

Bayesian Learning for Data-Efficient Control

Rowan McAllister, Apollo-University Of Cambridge Repository, Apollo-University Of Cambridge Repository
Third, we take a step towards data efficient learning of high-dimensional control using Bayesian neural networks (BNN).  ...  From robotics, to finance, to industrial processing, autonomous learning helps obviate a heavy reliance on experts for system identification and controller design.  ...  At time t = 0, the path ξ 0 is usually randomly initialised. Otherwise usually MPC initialises ξ t with shifted elements from the previously-optimised path ξ * t−1 .  ... 
doi:10.17863/cam.16688 fatcat:5gcifx6g3zcs3acdaxpqc4aseu

A complexity evolutionary theory for the emergence of financial and economic crises: Synchronising Gould and Minsky through von Neumann and Mandelbrot

Eduardo Viegas, Henrik Jeldtoft Jensen
CETFEC aims to identify the signals that lead the existence of the necessary conditions for the emergence of crises, rather than trying to predict the timing of crises.  ...  before its collapse in 2008.  ...  Targeted banks are chosen randomly.  ... 
doi:10.25560/44548 fatcat:iex5tbgpojf4xmu63xbgoabgba

Crawler [chapter]

Kenneth A. Ross, Christian S. Jensen, Richard Snodgrass, Curtis E. Dyreson, Christian S. Jensen, Richard Snodgrass, Spiros Skiadopoulos, Cristina Sirangelo, Mary Lynette Larsgaard, Gösta Grahne, Daniel Kifer, Hans-Arno Jacobsen (+106 others)
2009 Encyclopedia of Database Systems  
As a result, it is common (at the time of writing) for data accesses to RAM to require several hundred CPU cycles to resolve.  ...  For such workloads, improving the locality of data-intensive operations can have a direct impact on the system's overall performance.  ...  and run a batch process on it.  ... 
doi:10.1007/978-0-387-39940-9_2315 fatcat:x4qspjdytvhvroc7h753dihp7u

Hierarchical bayesian models for sparse signal recovery and sampling

Evripidis Karseras, Wei Dai, Kin Leung, European Commission
This includes extremely low-complexity algorithms for sparse recovery with a powerful analysis framework.  ...  Another innovation of this thesis are Bayesian models for signals whose components are known a priori to exhibit a certain statistical trend.  ...  Algorithm 9 AMP Algorithm for FFT basis. Initialise: r 0 = y, s 0 = 0.  ... 
doi:10.25560/32102 fatcat:j534y7z7sjh4pcmtpz5t7ljeoe

On the surprising capacity of linear combinations of embeddings for natural language processing [article]

Lyndon White
Acknowledgements Thank particularly to Christof Stocker, the creator of MLDatasets.jl (and numerous other packages), in particular for his bug reports, feature requests and code reviews; and for the initial  ...  Acknowledgements We would like to thank Dr Gerhard Wohlgenannt (ITMO University, Saint Petersburg) for his feedback on this work just prior to submission.  ...  This, like the word embeddings, is initialised randomly, then trained during the task.  ... 
doi:10.26182/w0c2-6887 fatcat:hacpoc35tne3pmcfy5wfdl6dsq

Dagstuhl Reports, Volume 3, Issue 09, September 2013, Complete Issue [article]

In the context of the Smart Grid, network overlays can play a key role, for enabling the management of resources in an adaptive fashion.  ...  from a Smart Grid metering network.  ...  For networks with small highway dimension, one can compute provably small hierarchical labels in polynomial time.  ... 
doi:10.4230/dagrep.3.9 fatcat:oilweqoffjbx7bjnxvg6vorrzu

Quality of Experience: From Assessment to Application (Dagstuhl Seminar 15022) Understanding Complexity in Multiobjective Optimization (Dagstuhl Seminar 15031) Model-driven Algorithms and Architectures for Self-Aware Computing Systems (Dagstuhl Seminar 15041) Artificial and Computational Intelligence in Games: Integration (Dagstuhl Seminar 15051)

Michael Philippsen, Pascal Felber, Michael Scott, J Eliot, Katrien De Moor, Markus Fiedler, Peter Reichl, Martín Varela, Salvatore Greco, Kathrin Klamroth, Joshua Knowles, Günter Rudolph (+13 others)
2015 unpublished
The example problems mentioned in this report may serve as a first benchmark for such approaches.  ...  = 1) to the most important (rank = m); select q criteria randomly with probabilities proportional to the ranks of criteria so that less important objectives have a higher probability of being omitted.  ...  The process worked reasonably well, but requires extensive hyper-parameter tuning on the following fronts: (a) neural network weight initialisation, (b) neural network learning rates, (c) exploration rates  ... 

An investigation of emergency department overcrowding using data mining and simulation : a patient treatment type perspective

Andrzej Stefan Ceglowski
Scientific Method was selected as an appropriate methodology for the research.  ...  This balance is often difficult to achieve and EDs become overwhelmed, resulting in long patient waits, overcrowded treatment areas and excessive stress for ED staff.  ...  If the populations are identical in location, the ranks should be randomly mixed between the two samples.  ... 
doi:10.4225/03/586dc33ad139b fatcat:odcko7xoj5ckbgplermrsv6t2q

Lisbon, Portugal VALID 2012 Editors VALID 2012 Committee VALID Advisory Chairs VALID 2012 Technical Progam Committee

Amir Alimohammad, Petre Dini, Amir Alimohammad, Sebastian Wieczorek, Eric Verhulst, Belgium Altreonic, Andrea Baruzzo, Amir Alimohammad, Abel Marrero, Sebastian Wieczorek, Eric Verhulst, Belgium Altreonic (+43 others)
We also gratefully thank the members of the VALID 2012 organizing committee for their help in handling the logistics and for their work that is making this professional meeting a success.  ...  Despite current solutions, virtualization and abstraction for large scale systems provide less visibility for vulnerability discovery and resolution, and make testing tedious, sometimes unsuccessful, if  ...  ACKNOWLEDGMENT The authors would like to thank Krishna Murthy Murlidhar, Sven Neuendorf and Jasmin Zieger for their contributions.  ...