Filters








58,162 Hits in 4.7 sec

Neural networks with late-phase weights [article]

Johannes von Oswald, Seijin Kobayashi, Alexander Meulemans, Christian Henning, Benjamin F. Grewe, João Sacramento
2022 arXiv   pre-print
These findings are complemented with a theoretical analysis of a noisy quadratic problem which provides a simplified picture of the late phases of neural network learning.  ...  Our results show that augmenting standard models with late-phase weights improves generalization in established benchmarks such as CIFAR-10/100, ImageNet and enwik8.  ...  Predictive accuracy is again highest for our neural networks with late-phase weights, trained with SGD or SWA, cf. Table 2 .  ... 
arXiv:2007.12927v4 fatcat:3gpxbwn625gmbgrk7jkes3xjwa

Neural networks with late-phase weights

Johannes Von Oswald, Seijin Kobayashi, João Sacramento, Alexander Meulemans, Christian Andreas Henning, Benjamin F Grewe
2021
These findings are complemented with a theoretical analysis of a noisy quadratic problem which provides a simplified picture of the late phases of neural network learning.  ...  Our results show that augmenting standard models with late-phase weights improves generalization in established benchmarks such as CIFAR-10/100, ImageNet and enwik8.  ...  Predictive accuracy is again highest for our neural networks with late-phase weights, trained with SGD or SWA, cf. Table 2 .  ... 
doi:10.5167/uzh-217757 fatcat:x2xgf6nm65d3lnqyfmstlpg5dq

Neural networks with late-phase weights

Johannes Von Oswald, Seijin Kobayashi, Alexander Meulemans, Christian Henning, Benjamin F. Grewe, João Sacramento
2021
These findings are complemented with a theoretical analysis of a noisy quadratic problem which provides a simplified picture of the late phases of neural network learning.  ...  Our results show that augmenting standard models with late-phase weights improves generalization in established benchmarks such as CIFAR-10/100, ImageNet and enwik8.  ...  Predictive accuracy is again highest for our neural networks with late-phase weights, trained with SGD or SWA, cf. Table 2 .  ... 
doi:10.3929/ethz-b-000524724 fatcat:xqjtfc75frenjbijnpqn45vczu

Detecting straggler MapReduce tasks in big data processing infrastructure by neural network

Amir Javadpour, Guojun Wang, Samira Rezaei, Kuan-Ching Li
2020 Journal of Supercomputing  
The proposed method is based on the application of a backpropagation Neural Network NN on the Hadoop for the detection of straggler tasks, to estimate the remaining execution time of tasks that is very  ...  Results achieved have been compared with popular algorithms in this domain such as LATE, ESAMR and the real remaining time for WordCount and Sort benchmarks, and shown able to detect straggler tasks and  ...  weight with the help of an artificial neural network algorithm in Map phase Learning rate = 0.05 Epoch = 100 Estimated weight with the help of an artificial neural network algorithm in reduce phase Progress  ... 
doi:10.1007/s11227-019-03136-6 fatcat:s52jcf4gqbhtnijpcrxmamqfnm

ANN-inspired Straggler Map Reduce Detection in Big Data Processing

Ajay Bansal, Manmohan Sharma, Ashu Gupta
2021 Converter  
The comparative analysis is done with some efficientmodels in this domain, such as LATE, ESAMR, and the real remaining time for WordCount and Sort benchmarks.  ...  Theproposed approach uses a backpropagation neural network on Hadoop to detect straggler tasks and calculate the remainingtask execution time, which is crucial in straggler task identification.  ...  neural network algorithm implementation method and LATE ESAMR Difference in the estimated runtime in mapping phase (Sort) Fig. 11 Difference in estimated run time in reduce phase (Sort) Table 1 1 The  ... 
doi:10.17762/converter.26 fatcat:hyhwxcebffhxhc4mwm27dwsw3y

Contrast-enhanced ultrasonography parameters in neural network diagnosis of liver tumors

Costin Teodor Streba
2012 World Journal of Gastroenterology  
The difference in maximum intensities, the time to reaching them and the aspect of the late/portal phase, as quantified by the neural network and a ratio between median intensities of the central and peripheral  ...  We registered wash-out in the late phase for most of the hypervascular metastases. Liver hemangiomas had contrast uptake in the arterial phase without agent wash-out in the portal-late phases.  ...  The difference in maximum intensities, the time to reaching them and the aspect of the late/portal phase, as quantified by the neural network and a ratio between median intensities of the central and peripheral  ... 
doi:10.3748/wjg.v18.i32.4427 pmid:22969209 pmcid:PMC3436061 fatcat:k2rll7fs2fbsviyj4kfnszcyru

Deep Learning Classification of Systemic Sclerosis Skin using the MobileNetV2 Model

Metin Akay, Yong Du, Cheryl L. Sershen, Minghua Wu, Ting Y. Chen, Shervin Assassi, Chandra Mohan, Yasemin M. Akay
2021 IEEE Open Journal of Engineering in Medicine and Biology  
The proposed network architecture consists of the UNet, a dense connectivity convolutional neural network (CNN) with added classifier layers that when combined with limited training data, yields better  ...  These results indicated that the MobileNetV2 architecture is more accurate and efficient compared to the CNN to classify normal, early and late phase SSc skin images.  ...  Training with a Pre-Trained Model In deep networks with many convolutional layers and different paths through the network, a good initialization of the weights is extremely important.  ... 
doi:10.1109/ojemb.2021.3066097 pmid:35402975 pmcid:PMC8901014 fatcat:shfkqrwitncv7aiyirkkeiqvxe

Topological Augmentation of Latent Information Streams in Feed-Forward Neural Networks [article]

James M Shine, Mike Li, Oluwasanmi Koyejo, Ben Fulcher, Joseph T Lizier
2020 bioRxiv   pre-print
Each phase brings the connections of the neural network into alignment with patterns of information contained in the input dataset, as well as the preceding layers.  ...  Our results enable a systems-level understanding of how deep neural networks function, and provide evidence of how neural networks reorganize edge weights and activity patterns so as to most effectively  ...  , four-layer, feed-forward neural network with no non-linearities was created with randomized weights (edge strengths: -1 to 1).  ... 
doi:10.1101/2020.09.30.321679 fatcat:vns64yheinduphegykdhphxzxu

Memory consolidation and improvement by synaptic tagging and capture in recurrent neural networks [article]

Jannik Luboeinski, Christian Tetzlaff
2020 bioRxiv   pre-print
We developed a theoretical model integrating the mechanisms underlying the STC hypothesis with calcium-based synaptic plasticity in a recurrent spiking neural network.  ...  This kind of memory enhancement can provide a new principle for storing information in biological and artificial neural circuits.  ...  in neural networks with parameters based on experimental findings (see Methods, [30, 77] ).  ... 
doi:10.1101/2020.05.08.084053 fatcat:glucjxlpsnh57iheeu3of7ja34

Construction and Application of the Online Finance Credit Risk Rating Model Based on the Artificial Neural Network

Yufeng Mao, Zongrun Wang, Xing Li, Chenggang Li, Hanning Wang, Ahmed Farouk
2021 Discrete Dynamics in Nature and Society  
In the late phase, there are few unlabeled samples. Hence, the backpropagation neural network (BPNN) was improved for online finance credit risk rating.  ...  This paper introduces the improved neural network technology to the credit risk rating of online finance.  ...  Figure 3 3 explains the idea of neural network optimization with the WPA improved by the belief learning model.  ... 
doi:10.1155/2021/6926216 fatcat:nh7prsl4nva6hpklwxyectnxay

Multi-Encoder Learning and Stream Fusion for Transformer-Based End-to-End Automatic Speech Recognition [article]

Timo Lohrenz, Zhengyang Li, Tim Fingscheidt
2021 arXiv   pre-print
network end-to-end model architectures.  ...  fusion of standard magnitude and phase features.  ...  Late Fusion As late fusion we investigate the fusion of output token probability vectors P mag ℓ and P phase ℓ stemming from separately trained transformer networks for each feature stream o T 1 and u  ... 
arXiv:2104.00120v2 fatcat:npqrlcu27zggpcunwda2lpgplu

Organization and priming of long-term memory representations with two-phase plasticity [article]

Jannik Luboeinski, Christian Tetzlaff
2021 bioRxiv   pre-print
To this end, we employ a biologically detailed neural network model of spiking neurons featuring STC, which models the learning and consolidation of long-term memory representations.  ...  Secondly, we find that hub-like structures counter this learning order effect for representations with less overlaps.  ...  for STC: the early-phase weight h ji , and the late-phase weight z ji .  ... 
doi:10.1101/2021.04.15.439982 fatcat:jdebnacklfhfxpvgty6hqnn64q

Page 250 of American Society of Civil Engineers. Collected Journals Vol. 12, Issue 4 [page]

1998 American Society of Civil Engineers. Collected Journals  
The forward pass calcu- lates the network output by propagating the input data through the network. The network output is then compared with the desired output to calculate the error.  ...  The life cycle of developing an ANN prototype as shown in Fig. 2 includes the following phases (Alsugair 1992): neural network justification, scope identification, training sets preparation, network structure  ... 

Learning Multimodal Gender Profile using Neural Networks

Carlos Pérez Estruch, Roberto Paredes, Paolo Rosso
2017 RANLP 2017 - Recent Advances in Natural Language Processing Meet Deep Learning  
The aim of this paper is to apply neural networks to perform data fusion, using an existing multimodal corpus, the NUS-MSS data set, that (not only) contains text data, but also image and location information  ...  Gender identification in social networks is one of the most popular aspects of user profile learning.  ...  (b) Late fusion architecture Figure 1 : Artificial neural network data fusion strategies. In (a) we can see the basic early fusion topology. In (b) we have the late fusion neural network.  ... 
doi:10.26615/978-954-452-049-6_075 dblp:conf/ranlp/EstruchPR17 fatcat:x4qyrsf6wnemnhswyrcatud2m4

The Study of Adoption of Neural Network Approach in Fingerprint Recognition

Divyakant T.Meva, C. K. Kumbharana, Amit D. Kothari
2012 International Journal of Computer Applications  
The network of interconnected neurons is known as neural network. A neural network is composed of a number of nodes, or units, connected by links. Each link has a numeric weight associated with it.  ...  It reduces total number of comparisons during the matching phase and time is also reduced. NEURAL NETWORK Neurons are biological elements present in the human brain.  ... 
doi:10.5120/5007-7326 fatcat:znlfwha3hvgxpitr2dbrbrzt2m
« Previous Showing results 1 — 15 out of 58,162 results