Filters








220 Hits in 0.74 sec

Toxicity Prediction using Deep Learning [article]

Thomas Unterthiner, Andreas Mayr, Günter Klambauer, Sepp Hochreiter
<span title="2015-03-04">2015</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Dahl's success inspired us to use Deep Learning for toxicity and target prediction (Unterthiner et al., 2014) .  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1503.01445v1">arXiv:1503.01445v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/hp6kwycg5ja7pfq56qb6bnkhwu">fatcat:hp6kwycg5ja7pfq56qb6bnkhwu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200827152140/https://arxiv.org/pdf/1503.01445v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/52/e0/52e0d621336e891223f18e0ff0aee2a7982ca823.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1503.01445v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Rectified Factor Networks [article]

Djork-Arné Clevert, Andreas Mayr, Thomas Unterthiner, Sepp Hochreiter
<span title="2015-06-11">2015</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
We propose rectified factor networks (RFNs) to efficiently construct very sparse, non-linear, high-dimensional representations of the input. RFN models identify rare and small events in the input, have a low interference between code units, have a small reconstruction error, and explain the data covariance structure. RFN learning is a generalized alternating minimization algorithm derived from the posterior regularization method which enforces non-negative and normalized posterior means. We
more &raquo; ... f convergence and correctness of the RFN learning algorithm. On benchmarks, RFNs are compared to other unsupervised methods like autoencoders, RBMs, factor analysis, ICA, and PCA. In contrast to previous sparse coding methods, RFNs yield sparser codes, capture the data's covariance structure more precisely, and have a significantly smaller reconstruction error. We test RFNs as pretraining technique for deep networks on different vision datasets, where RFNs were superior to RBMs and autoencoders. On gene expression data from two pharmaceutical drug discovery studies, RFNs detected small and rare gene modules that revealed highly relevant new biological insights which were so far missed by other unsupervised methods.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1502.06464v2">arXiv:1502.06464v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/vkbozsi2hbfz7jc2nm2dochg6q">fatcat:vkbozsi2hbfz7jc2nm2dochg6q</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200930140713/https://arxiv.org/pdf/1502.06464v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/da/e5/dae5c921ccca649ee89f9b31cae2c0aa25b213c2.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1502.06464v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Object-Centric Learning with Slot Attention [article]

Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, Thomas Kipf
<span title="2020-10-14">2020</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
In Advances in Neural Information Processing Systems, pages 3207-3217, 2019. [32] Nicola De Cao and Thomas Kipf.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2006.15055v2">arXiv:2006.15055v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/2uvpbz754nftbnizjin6zeiyai">fatcat:2uvpbz754nftbnizjin6zeiyai</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20201020101212/https://arxiv.org/pdf/2006.15055v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2006.15055v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

DeepTox: Toxicity prediction using deep learning

Günter Klambauer, Thomas Unterthiner, Andreas Mayr, Sepp Hochreiter
<span title="">2017</span> <i title="Elsevier BV"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/4dxjv2b365cpnfqwmsabxwvday" style="color: black;">Toxicology Letters</a> </i> &nbsp;
., 2014; Unterthiner et al., 2014 Unterthiner et al., , 2015 Ma et al., 2015) , which made it our prime candidate for toxicity prediction.  ...  GPUs alleviate the problem of large computation times, typically by using CUDA kernels on Nvidia cards (Raina et al., 2009; Unterthiner et al., 2014 Unterthiner et al., , 2015 Clevert et al., 2015) .  ...  Copyright © 2016 Mayr, Klambauer, Unterthiner and Hochreiter. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY).  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/j.toxlet.2017.07.175">doi:10.1016/j.toxlet.2017.07.175</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/rajiojv6fjeornjxgdjmf5e5iu">fatcat:rajiojv6fjeornjxgdjmf5e5iu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20171007050514/http://publisher-connector.core.ac.uk/resourcesync/data/Frontiers/pdf/8a7/aHR0cDovL2pvdXJuYWwuZnJvbnRpZXJzaW4ub3JnL2FydGljbGUvMTAuMzM4OS9mZW52cy4yMDE1LjAwMDgwL3BkZg%3D%3D.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/1d/29/1d298139fc73654db7d8d936b07fa9e85bba008c.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/j.toxlet.2017.07.175"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> elsevier.com </button> </a>

First Order Generative Adversarial Networks [article]

Calvin Seward, Thomas Unterthiner, Urs Bergmann, Nikolay Jetchev, Sepp Hochreiter
<span title="2018-06-07">2018</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
., 2017) and (Unterthiner et al., 2018) .  ...  Methods other than Coulomb GAN (Unterthiner et al., 2018) WGAN-GP (Gulrajani et al., 2017; Heusel et al., 2017) and the Sobolev GAN (Mroueh et al., 2018) have not been shown to be successful at this  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1802.04591v2">arXiv:1802.04591v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/vogvm3pr7jcdjalfltmvjl2q3a">fatcat:vogvm3pr7jcdjalfltmvjl2q3a</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200929022959/https://arxiv.org/pdf/1802.04591v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/4c/5d/4c5d132b11aedbecb3ed0b7c1c915385d9f7a388.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1802.04591v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

DeepTox: Toxicity Prediction using Deep Learning

Andreas Mayr, Günter Klambauer, Thomas Unterthiner, Sepp Hochreiter
<span title="2016-02-02">2016</span> <i title="Frontiers Media SA"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/jd2zpnfbhfdcjmnqcecunt4hou" style="color: black;">Frontiers in Environmental Science</a> </i> &nbsp;
., 2014; Unterthiner et al., 2014 Unterthiner et al., , 2015 Ma et al., 2015) , which made it our prime candidate for toxicity prediction.  ...  GPUs alleviate the problem of large computation times, typically by using CUDA kernels on Nvidia cards (Raina et al., 2009; Unterthiner et al., 2014 Unterthiner et al., , 2015 Clevert et al., 2015) .  ...  Copyright © 2016 Mayr, Klambauer, Unterthiner and Hochreiter. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY).  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.3389/fenvs.2015.00080">doi:10.3389/fenvs.2015.00080</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/f3uxqnexobf43jg5q2z6rn7pqq">fatcat:f3uxqnexobf43jg5q2z6rn7pqq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20180205073220/http://www.bioinf.jku.at:80/publications/2016/fenvs-03-00080.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/9a/5e/9a5e09c5871cda34598316d79b26fe38e2d2a37d.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.3389/fenvs.2015.00080"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> frontiersin.org </button> </a>

Interpretable Deep Learning in Drug Discovery [article]

Kristina Preuer, Günter Klambauer, Friedrich Rippmann, Sepp Hochreiter, Thomas Unterthiner
<span title="2019-03-18">2019</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Without any means of interpretation, neural networks that predict molecular properties and bioactivities are merely black boxes. We will unravel these black boxes and will demonstrate approaches to understand the learned representations which are hidden inside these models. We show how single neurons can be interpreted as classifiers which determine the presence or absence of pharmacophore- or toxicophore-like structures, thereby generating new insights and relevant knowledge for chemistry,
more &raquo; ... macology and biochemistry. We further discuss how these novel pharmacophores/toxicophores can be determined from the network by identifying the most relevant components of a compound for the prediction of the network. Additionally, we propose a method which can be used to extract new pharmacophores from a model and will show that these extracted structures are consistent with literature findings. We envision that having access to such interpretable knowledge is a crucial aid in the development and design of new pharmaceutically active molecules, and helps to investigate and understand failures and successes of current methods.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1903.02788v2">arXiv:1903.02788v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/ub5tevhaanct7evmvd5nnfuphy">fatcat:ub5tevhaanct7evmvd5nnfuphy</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200905162856/https://arxiv.org/pdf/1903.02788v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/18/c0/18c0d20a56f2e0a4700d0cdbb843b4b45c6f57ef.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1903.02788v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Rectified factor networks for biclustering of omics data

Djork-Arné Clevert, Thomas Unterthiner, Gundula Povysil, Sepp Hochreiter
<span title="2017-07-12">2017</span> <i title="Oxford University Press (OUP)"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/wmo54ba2jnemdingjj4fl3736a" style="color: black;">Bioinformatics</a> </i> &nbsp;
Motivation: Biclustering has become a major tool for analyzing large datasets given as matrix of samples times features and has been successfully applied in life sciences and e-commerce for drug design and recommender systems, respectively. Factor Analysis for Bicluster Acquisition (FABIA), one of the most successful biclustering methods, is a generative model that represents each bicluster by two sparse membership vectors: one for the samples and one for the features. However, FABIA is
more &raquo; ... ed to about 20 code units because of the high computational complexity of computing the posterior. Furthermore, code units are sometimes insufficiently decorrelated and sample membership is difficult to determine. We propose to use the recently introduced unsupervised Deep Learning approach Rectified Factor Networks (RFNs) to overcome the drawbacks of existing biclustering methods. RFNs efficiently construct very sparse, non-linear, high-dimensional representations of the input via their posterior means. RFN learning is a generalized alternating minimization algorithm based on the posterior regularization method which enforces non-negative and normalized posterior means. Each code unit represents a bicluster, where samples for which the code unit is active belong to the bicluster and features that have activating weights to the code unit belong to the bicluster. Results: On 400 benchmark datasets and on three gene expression datasets with known clusters, RFN outperformed 13 other biclustering methods including FABIA. On data of the 1000 Genomes Project, RFN could identify DNA segments which indicate, that interbreeding with other hominins starting already before ancestors of modern humans left Africa. Availability and implementation: https://github.com/
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1093/bioinformatics/btx226">doi:10.1093/bioinformatics/btx226</a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pubmed/28881961">pmid:28881961</a> <a target="_blank" rel="external noopener" href="https://pubmed.ncbi.nlm.nih.gov/PMC5870657/">pmcid:PMC5870657</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/lg673yjrs5gajmx32ftauplfca">fatcat:lg673yjrs5gajmx32ftauplfca</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20191026202925/http://europepmc.org/backend/ptpmcrender.fcgi?accid=PMC5870657&amp;blobtype=pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/44/88/4488f2978411bf6886bbe8e2fe4219c2975f18de.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1093/bioinformatics/btx226"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> oup.com </button> </a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5870657" title="pubmed link"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> pubmed.gov </button> </a>

Understanding Robustness of Transformers for Image Classification [article]

Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, Andreas Veit
<span title="2021-10-08">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf.  ...  In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision Workshops, pages 0-0, 2019. 2 [31] 86(11):2278-2324, 1998. 1 [32] Francesco Locatello, Dirk Weissenborn, Thomas Un- terthiner  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2103.14586v2">arXiv:2103.14586v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/wbouwz4rbfh3vj7qvjgxpe24be">fatcat:wbouwz4rbfh3vj7qvjgxpe24be</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20211012190725/https://arxiv.org/pdf/2103.14586v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/d2/a3/d2a3bb6356d439146cd8d8e72dc728a1e3d93e7f.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2103.14586v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

GradMax: Growing Neural Networks using Gradient Information [article]

Utku Evci, Bart van Merriënboer, Thomas Unterthiner, Max Vladymyrov, Fabian Pedregosa
<span title="2022-02-23">2022</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
The architecture and the parameters of neural networks are often optimized independently, which requires costly retraining of the parameters whenever the architecture is modified. In this work we instead focus on growing the architecture without requiring costly retraining. We present a method that adds new neurons during training without impacting what is already learned, while improving the training dynamics. We achieve the latter by maximizing the gradients of the new weights and find the
more &raquo; ... imal initialization efficiently by means of the singular value decomposition (SVD). We call this technique Gradient Maximizing Growth (GradMax) and demonstrate its effectiveness in variety of vision tasks and architectures.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2201.05125v2">arXiv:2201.05125v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/ht7gx5ju65eprbxzi3k2zxicum">fatcat:ht7gx5ju65eprbxzi3k2zxicum</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20220520080100/https://arxiv.org/pdf/2201.05125v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/b8/a2/b8a2baf988f4fc4a466d5199ced7a17f2d3b8759.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2201.05125v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Do Vision Transformers See Like Convolutional Neural Networks? [article]

Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, Alexey Dosovitskiy
<span title="2022-03-03">2022</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Convolutional neural networks (CNNs) have so far been the de-facto model for visual data. Recent work has shown that (Vision) Transformer models (ViT) can achieve comparable or even superior performance on image classification tasks. This raises a central question: how are Vision Transformers solving these tasks? Are they acting like convolutional networks, or learning entirely different visual representations? Analyzing the internal representation structure of ViTs and CNNs on image
more &raquo; ... ion benchmarks, we find striking differences between the two architectures, such as ViT having more uniform representations across all layers. We explore how these differences arise, finding crucial roles played by self-attention, which enables early aggregation of global information, and ViT residual connections, which strongly propagate features from lower to higher layers. We study the ramifications for spatial localization, demonstrating ViTs successfully preserve input spatial information, with noticeable effects from different classification methods. Finally, we study the effect of (pretraining) dataset scale on intermediate features and transfer learning, and conclude with a discussion on connections to new architectures such as the MLP-Mixer.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2108.08810v2">arXiv:2108.08810v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/nju5i5wbbncavpit3pi2gcbsoe">fatcat:nju5i5wbbncavpit3pi2gcbsoe</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20220309142325/https://arxiv.org/pdf/2108.08810v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/39/b4/39b492db00faead70bc3f4fb4b0364d94398ffdb.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2108.08810v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Self-Normalizing Neural Networks [article]

Günter Klambauer, Thomas Unterthiner, Andreas Mayr, Sepp Hochreiter
<span title="2017-09-07">2017</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Deep Learning has revolutionized vision via convolutional neural networks (CNNs) and natural language processing via recurrent neural networks (RNNs). However, success stories of Deep Learning with standard feed-forward neural networks (FNNs) are rare. FNNs that perform well are typically shallow and, therefore cannot exploit many levels of abstract representations. We introduce self-normalizing neural networks (SNNs) to enable high-level abstract representations. While batch normalization
more &raquo; ... res explicit normalization, neuron activations of SNNs automatically converge towards zero mean and unit variance. The activation function of SNNs are "scaled exponential linear units" (SELUs), which induce self-normalizing properties. Using the Banach fixed-point theorem, we prove that activations close to zero mean and unit variance that are propagated through many network layers will converge towards zero mean and unit variance -- even under the presence of noise and perturbations. This convergence property of SNNs allows to (1) train deep networks with many layers, (2) employ strong regularization, and (3) to make learning highly robust. Furthermore, for activations not close to unit variance, we prove an upper and lower bound on the variance, thus, vanishing and exploding gradients are impossible. We compared SNNs on (a) 121 tasks from the UCI machine learning repository, on (b) drug discovery benchmarks, and on (c) astronomy tasks with standard FNNs and other machine learning methods such as random forests and support vector machines. SNNs significantly outperformed all competing FNN methods at 121 UCI tasks, outperformed all competing methods at the Tox21 dataset, and set a new record at an astronomy data set. The winning SNN architectures are often very deep. Implementations are available at: github.com/bioinf-jku/SNNs.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1706.02515v5">arXiv:1706.02515v5</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/ynvna3ldinfcnms4p4l75od5wa">fatcat:ynvna3ldinfcnms4p4l75od5wa</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200914084742/https://arxiv.org/pdf/1706.02515v5.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/e7/02/e702ff10007617aa8f925236a4d71c921c42b4fa.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1706.02515v5" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Predicting Neural Network Accuracy from Weights [article]

Thomas Unterthiner, Daniel Keysers, Sylvain Gelly, Olivier Bousquet, Ilya Tolstikhin
<span title="2021-04-09">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
We show experimentally that the accuracy of a trained neural network can be predicted surprisingly well by looking only at its weights, without evaluating it on input data. We motivate this task and introduce a formal setting for it. Even when using simple statistics of the weights, the predictors are able to rank neural networks by their performance with very high accuracy (R2 score more than 0.98). Furthermore, the predictors are able to rank networks trained on different, unobserved datasets
more &raquo; ... and with different architectures. We release a collection of 120k convolutional neural networks trained on four different datasets to encourage further research in this area, with the goal of understanding network training and performance better.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2002.11448v4">arXiv:2002.11448v4</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/y4kcnejpefc6xcbznbl3o55iam">fatcat:y4kcnejpefc6xcbznbl3o55iam</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200605140334/https://arxiv.org/pdf/2002.11448v3.pdf" title="fulltext PDF download [not primary version]" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <span style="color: #f43e3e;">&#10033;</span> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2002.11448v4" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

DEXUS: identifying differential expression in RNA-Seq studies with unknown conditions

Günter Klambauer, Thomas Unterthiner, Sepp Hochreiter
<span title="2013-09-17">2013</span> <i title="Oxford University Press (OUP)"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/hfp6p6inqbdexbsu4r7usndpte" style="color: black;">Nucleic Acids Research</a> </i> &nbsp;
Detection of differential expression in RNA-Seq data is currently limited to studies in which two or more sample conditions are known a priori. However, these biological conditions are typically unknown in cohort, cross-sectional and nonrandomized controlled studies such as the HapMap, the ENCODE or the 1000 Genomes project. We present DEXUS for detecting differential expression in RNA-Seq data for which the sample conditions are unknown. DEXUS models read counts as a finite mixture of negative
more &raquo; ... binomial distributions in which each mixture component corresponds to a condition. A transcript is considered differentially expressed if modeling of its read counts requires more than one condition. DEXUS decomposes read count variation into variation due to noise and variation due to differential expression. Evidence of differential expression is measured by the informative/noninformative (I/NI) value, which allows differentially expressed transcripts to be extracted at a desired specificity (significance level) or sensitivity (power). DEXUS performed excellently in identifying differentially expressed transcripts in data with unknown conditions. On 2400 simulated data sets, I/NI value thresholds of 0.025, 0.05 and 0.1 yielded average specificities of 92, 97 and 99% at sensitivities of 76, 61 and 38%, respectively. On real-world data sets, DEXUS was able to detect differentially expressed transcripts related to sex, species, tissue, structural variants or quantitative trait loci. The DEXUS R package is publicly available from Bioconductor and the scripts for all experiments are available at
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1093/nar/gkt834">doi:10.1093/nar/gkt834</a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pubmed/24049071">pmid:24049071</a> <a target="_blank" rel="external noopener" href="https://pubmed.ncbi.nlm.nih.gov/PMC3834838/">pmcid:PMC3834838</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/kshdbtwz7rfozbyhki4v4wauaa">fatcat:kshdbtwz7rfozbyhki4v4wauaa</a> </span>
<a target="_blank" rel="noopener" href="https://archive.org/download/pubmed-PMC3834838/PMC3834838-gkt834.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> File Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/61/4e/614e046f748def7594758d807801d309dac24f52.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1093/nar/gkt834"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> oup.com </button> </a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3834838" title="pubmed link"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> pubmed.gov </button> </a>

Detection of viral sequence fragments of HIV-1 subfamilies yet unknown

Thomas Unterthiner, Anne-Kathrin Schultz, Jan Bulla, Burkhard Morgenstern, Mario Stanke, Ingo Bulla
<span title="">2011</span> <i title="Springer Nature"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/n5zrklrhlzhtdorf4rk4rmeo3i" style="color: black;">BMC Bioinformatics</a> </i> &nbsp;
Acknowledgements We would like to thank Thomas Leitner for the encouragement to develop USF and Heinrich Hering for proofreading.  ...  Unterthiner et al. BMC Bioinformatics 2011, 12:93 http://www.biomedcentral.com/1471-2105/12/93 Page 5 of 13  ...  Unterthiner et al. BMC Bioinformatics 2011, 12:93 http://www.biomedcentral.com/1471-2105/12/93 · (0.34, 0.66) + 2 3 · (0.92, 0.08) = (0.72, 0.28) as estimate for the emission probabilities.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1186/1471-2105-12-93">doi:10.1186/1471-2105-12-93</a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pubmed/21481263">pmid:21481263</a> <a target="_blank" rel="external noopener" href="https://pubmed.ncbi.nlm.nih.gov/PMC3086866/">pmcid:PMC3086866</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/je457nzoqrbz7l7gmruslypkui">fatcat:je457nzoqrbz7l7gmruslypkui</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170809104339/http://virion.ucsd.edu/Jclub2011-06-03.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/95/11/9511bcca32e439bf30483a6bfe8b542513b77238.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1186/1471-2105-12-93"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> springer.com </button> </a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3086866" title="pubmed link"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> pubmed.gov </button> </a>
&laquo; Previous Showing results 1 &mdash; 15 out of 220 results