Filters








528 Hits in 8.8 sec

SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models [article]

Zaccharie Ramzi, Florian Mannel, Shaojie Bai, Jean-Luc Starck, Philippe Ciuciu, Thomas Moreau
2022 arXiv   pre-print
The main idea is to use the quasi-Newton matrices from the forward pass to efficiently approximate the inverse Jacobian matrix in the direction needed for the gradient computation.  ...  In Deep Equilibrium Models (DEQs), the training is performed as a bi-level problem, and its computational complexity is partially driven by the iterative inversion of a huge Jacobian matrix.  ...  SHINE Roughly speaking, our proposition is to use B −1 = lim n→∞ B −1 n as a replacement for J g θ (z ) −1 in (3), i.e. to share the inverse estimate between the forward and the backward passes.  ... 
arXiv:2106.00553v3 fatcat:lcnqmu6u2vfjrezf3fomaefp54

Learned reconstruction methods with convergence guarantees [article]

Subhadip Mukherjee, Andreas Hauptmann, Ozan Öktem, Marcelo Pereyra, Carola-Bibiane Schönlieb
2022 arXiv   pre-print
This has catalyzed an ongoing quest for precise characterization of correctness and reliability of data-driven methods in critical use-cases, for instance in medical imaging.  ...  An example that is highlighted is the role of ICNN, offering the possibility to combine the power of deep learning with classical convex regularization theory for devising methods that are provably convergent  ...  compute the forward pass of DEQ models and used the QN matrices (that are available as a bi-product of the forward pass) to approximate the inverse Jacobian.  ... 
arXiv:2206.05431v2 fatcat:4cz56hdgxjg47igzn62zrzcvea

State-of-the-Art in the Architecture, Methods and Applications of StyleGAN [article]

Amit H. Bermano and Rinon Gal and Yuval Alaluf and Ron Mokady and Yotam Nitzan and Omer Tov and Or Patashnik and Daniel Cohen-Or
2022 arXiv   pre-print
It aims to be of use for both newcomers, who wish to get a grasp of the field, and for more experienced readers that might benefit from seeing current research trends and existing tools laid out.  ...  Looking forward, we point out StyleGAN's limitations and speculate on current trends and promising directions for future research, such as task and target specific fine-tuning.  ...  We refer the reader to Figure 9 for a comparison of various optimization-based and encoder-based inversion techniques.  ... 
arXiv:2202.14020v1 fatcat:qu3plbdnszdujcwxwq3zizysje

Shining Light on Manure Improves Livestock and Land Management Graphic Design and Layout

John Walker
unpublished
Peak estimated CP and DOM occurred in April with another minor peak in September for CP. These peaks are consistent with the bi-modal rainfall pattern for this region.  ...  Combinations arise from the sharing of NIRS energy between two or more fundamental absorptions.  ...  Alvarez (1994) used morphology to distinguish between fecal pellets of male and female red and fallow deer. The success rates for visual identification ranged from 60.0 to 80.0% in this study.  ... 
fatcat:oyepzomaevcrlflnrxcy3tv4jq

ProtTrans: Towards Cracking the Language of Lifes Code Through Self-Supervised Deep Learning and High Performance Computing

Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Wang Yu, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, Debsindhu Bhowmik, Burkhard Rost
2021 IEEE Transactions on Pattern Analysis and Machine Intelligence  
Computational biology and bioinformatics provide vast data gold-mines from protein sequences, ideal for Language Models taken from NLP.  ...  Here, we trained two auto-regressive models (Transformer-XL, XLNet) and four auto-encoder models (BERT, Albert, Electra, T5) on data from UniRef and BFD containing up to 393 billion amino acids.  ...  Furthermore, thanks to both Adam Roberts and Colin Raffel for help with the T5 model.  ... 
doi:10.1109/tpami.2021.3095381 pmid:34232869 fatcat:tkltzxgdsveyxpqvfgogtnenhi

Training Spiking Neural Networks Using Lessons From Deep Learning [article]

Jason K. Eshraghian and Max Ward and Emre Neftci and Xinxin Wang and Gregor Lenz and Girish Dwivedi and Mohammed Bennamoun and Doo Seok Jeong and Wei D. Lu
2022 arXiv   pre-print
This paper serves as a tutorial and perspective showing how to apply the lessons learnt from several decades of research in deep learning, gradient descent, backpropagation and neuroscience to biologically  ...  The brain is the perfect place to look for inspiration to develop more efficient neural networks.  ...  Acknowledgements We would like to thank Sumit Bam Shrestha, Garrick Orchard, and Albert Albesa González for their insightful discussions over the course of putting together this paper, and iDataMap Corporation  ... 
arXiv:2109.12894v4 fatcat:zujzdtzaijak5bklbqufrxr57q

Backwards is the way forward: Feedback in the cortical hierarchy predicts the expected future

Lars Muckli, Lucy S. Petro, Fraser W. Smith
2013 Behavioral and Brain Sciences  
Second, hierarchical prediction offers progress on aconcretedescriptive level for testing and constraining conceptual elements and mechanisms of predictive coding models (estimation of predictions, prediction  ...  errors, and internal models).  ...  The implicit inversion of a generative model happens when prediction error is minimized between the model maintained in the brain and the sensory input (how the world impinges on the senses).  ... 
doi:10.1017/s0140525x12002361 pmid:23663531 fatcat:d7iicta6qfau5cb3znkiwtk4g4

Commonsense Knowledge Base Completion with Structural and Semantic Context

Chaitanya Malaviya, Chandra Bhagavatula, Antoine Bosselut, Yejin Choi
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
Further analysis on model predictions shines light on the types of commonsense knowledge that language models capture well.  ...  Our results demonstrate the effectiveness of language model representations in boosting link prediction performance and the advantages of learning from local graph structure (+1.5 points in MRR for ConceptNet  ...  Our work shares the high-level spirit of recent work from Petroni et al.  ... 
doi:10.1609/aaai.v34i03.5684 fatcat:klu7rkg6zvf3pacnklms3jf7ua

Whatever next? Predictive brains, situated agents, and the future of cognitive science

2013 Behavioral and Brain Sciences  
Such accounts offer a unifying model of perception and action, illuminate the functional role of attention, and may neatly capture the special contribution of cortical processing to adaptive success.  ...  Sections 1 and 2 lay out the key elements and implications of the approach.  ...  The implicit inversion of a generative model happens when prediction error is minimized between the model maintained in the brain and the sensory input (how the world impinges on the senses).  ... 
doi:10.1017/s0140525x12000477 pmid:23663408 fatcat:k7kqhjt5vngdfjftnixoef5bxq

Attention is more than prediction precision

Howard Bowman, Marco Filetti, Brad Wyble, Christian Olivers
2013 Behavioral and Brain Sciences  
AbstractA cornerstone of the target article is that, in a predictive coding framework, attention can be modelled by weighting prediction error with a measure of precision.  ...  We argue that this is not a complete explanation, especially in the light of ERP (event-related potentials) data showing large evoked responses for frequently presented target stimuli, which thus are predicted  ...  The implicit inversion of a generative model happens when prediction error is minimized between the model maintained in the brain and the sensory input (how the world impinges on the senses).  ... 
doi:10.1017/s0140525x12002324 pmid:23663435 fatcat:hrinlk77vbgtnifunk5tdkhn5i

Are we predictive engines? Perils, prospects, and the puzzle of the porous perceiver

Andy Clark
2013 Behavioral and Brain Sciences  
In assessing the attractions and pitfalls of the proposal we should keep that element distinct from larger, though interlocking, issues concerning the nature of adaptive organization in general.  ...  AbstractThe target article sketched and explored a mechanism (action-oriented predictive processing) most plausibly associated with core forms of cortical processing.  ...  The implicit inversion of a generative model happens when prediction error is minimized between the model maintained in the brain and the sensory input (how the world impinges on the senses).  ... 
doi:10.1017/s0140525x12002440 pmid:23814868 fatcat:7dwe5kt5hzeztegieyetaronfa

Modern applications of machine learning in quantum sciences [article]

Anna Dawid, Julian Arnold, Borja Requena, Alexander Gresch, Marcin Płodzień, Kaelan Donatella, Kim A. Nicoli, Paolo Stornati, Rouven Koch, Miriam Büttner, Robert Okuła, Gorka Muñoz-Gil (+17 others)
2022 arXiv   pre-print
control, and quantum circuits optimization.  ...  We cover the use of deep learning and kernel methods in supervised, unsupervised, and reinforcement learning algorithms for phase classification, representation of many-body quantum states, quantum feedback  ...  Briegel, Lorenzo Cardarelli, Kacper Cybiński, and Mario Krenn for useful discussions and Fesido Studio Graficzne for the graphical design of the Lecture Notes.  ... 
arXiv:2204.04198v2 fatcat:slojwtqwfzgbfgvz3pssdkwhtm

Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigation [article]

Yicong Hong, Zun Wang, Qi Wu, Stephen Gould
2022 arXiv   pre-print
discrete-to-continuous gap by 11.76% Success Weighted by Path Length (SPL) for the Cross-Modal Matching Agent and 18.24% SPL for the Recurrent VLN-BERT.  ...  of navigation with low-level controls to jumping from node to node with high-level actions by grounding to an image of a navigable direction.  ...  We set the maximum number of prediction from each heatmap to be 5. All models are trained on a NVIDIA 3090 GPU with a learning rate of 10 ´6 and batch size 64 using the AdamW optimizer [36] .  ... 
arXiv:2203.02764v1 fatcat:2bnzgteuprbzvd3cw2x363bpbu

The AROME-France Convective-Scale Operational Model

Y. Seity, P. Brousseau, S. Malardel, G. Hello, P. Bénard, F. Bouttier, C. Lac, V. Masson
2011 Monthly Weather Review  
The performance of the forecast model is evaluated using objective scores and case studies that highlight its benefits and weaknesses.  ...  This paper presents the main characteristics of this new numerical weather prediction system: the nonhydrostatic dynamical model core, detailed moist physics, and the associated three-dimensional variational  ...  Maziejewski for his help in revising the manuscript.  ... 
doi:10.1175/2010mwr3425.1 fatcat:4up2xtyayvb2vlaiutivdytezu

Commonsense Knowledge Base Completion with Structural and Semantic Context [article]

Chaitanya Malaviya, Chandra Bhagavatula, Antoine Bosselut, Yejin Choi
2019 arXiv   pre-print
Further analysis on model predictions shines light on the types of commonsense knowledge that language models capture well.  ...  Our results demonstrate the effectiveness of language model representations in boosting link prediction performance and the advantages of learning from local graph structure (+1.5 points in MRR for ConceptNet  ...  Acknowledgments We thank the anonymous reviewers and the Mosaic team at AI2 for their insightful comments and suggestions.  ... 
arXiv:1910.02915v2 fatcat:qkhgtbjczjgwxamozhzj5jp2am
« Previous Showing results 1 — 15 out of 528 results