3,375 Hits in 6.5 sec

Neural Path Features and Neural Path Kernel : Understanding the role of gates in deep learning [article]

Chandrashekar Lakshminarayanan, Amit Vikram Singh
2021 arXiv   pre-print
To this end, we encode the on/off state of the gates of a given input in a novel 'neural path feature' (NPF), and the weights of the DNN are encoded in a novel 'neural path value' (NPV).  ...  In this paper, we analytically characterise the role of active sub-networks in deep learning.  ...  We would also like to thank Indian Institute of Technology Palakkad for the 'Seed Grant', and Science and Engineering Research Board (SERB), Department of Science and Technology, Government of India for  ... 
arXiv:2006.10529v2 fatcat:fq3asbydtrabvbuzefjmkgbjge

Explicitising The Implicit Intrepretability of Deep Neural Networks Via Duality [article]

Chandrashekar Lakshminarayanan, Amit Vikram Singh, Arun Rajkumar
2022 arXiv   pre-print
It was shown that (i) the information in the gates is analytically characterised by a kernel called the neural path kernel (NPK) and (ii) most critical information is learnt in the gates, in that, given  ...  Recent work by Lakshminarayanan and Singh [2020] provided a dual view for fully connected deep neural networks (DNNs) with rectified linear units (ReLU).  ...  the DNN, and (ii) in the limit of infinite width, learning the weights with fixed gates is equal to a kernel method with the neural path kernel (NPK) 1 which is the kernel associated with the neural path  ... 
arXiv:2203.16455v1 fatcat:sevasqw67jfcpifb2pzlgz3w3i

Disentangling deep neural networks with rectified linear units using duality [article]

Chandrashekar Lakshminarayanan, Amit Vikram Singh
2021 arXiv   pre-print
via the so called neural path kernel (NPK) which depends on inputs and gates.  ...  We extend the recently developed dual view in which the computation is broken path-wise to show that learning in the gates is more crucial, and learning the weights given the gates is characterised analytically  ...  To understand the role of the gates, a deep gated network (DGN) (see Figure 1 ) was used to disentangle the learning in the gates from the learning in weights.  ... 
arXiv:2110.03403v1 fatcat:xrgv7eplbrbdloqxcfjed7e2sq

Understanding a Deep Neural Network Based on Neural-Path Coding

Dawei Dai, Chengfu Tang, Dongyao Zhao, Shuyin Xia, Haoyue Bai
2020 IEEE Access  
Recently, the deep neural network has achieved considerable success in the field of machine learning.  ...  First, for a trained neural model, we quantify the neural-path in which the role of a neuron was assumed to control the amount of information that can be passed through; Second, we define a Euclidean distance  ...  and when can I trust the agent? FIGURE 1 . 1 Overview of neural-path in a deep neural network.  ... 
doi:10.1109/access.2020.3024959 fatcat:xpvig75izzej3h7cvf4b35u6du

The Neural Race Reduction: Dynamics of Abstraction in Gated Networks [article]

Andrew M. Saxe, Shagun Sodhani, Sam Lewallen
2022 arXiv   pre-print
the role of modularity and compositionality in solving real-world problems.  ...  Our theoretical understanding of deep learning has not kept pace with its empirical success.  ...  A.S. is a CIFAR Azrieli Global Scholar in the Learning in Machines & Brains program.  ... 
arXiv:2207.10430v1 fatcat:ye7zamhdqre5leldby2vmqb7ru

The Neural Race Reduction: Dynamics of Abstraction in Gated Networks

Andrew Saxe, Shagun Sodhani, Sam Jay Lewallen
2022 International Conference on Machine Learning  
the role of modularity and compositionality in solving real-world problems.  ...  Our theoretical understanding of deep learning has not kept pace with its empirical success.  ...  A.S. is a CIFAR Azrieli Global Scholar in the Learning in Machines & Brains program.  ... 
dblp:conf/icml/SaxeSL22 fatcat:yoy6zyqulfacbgikb6mqjqcupi

Deep Learning Based Brain Tumor Segmentation: A Survey [article]

Zhihua Liu, Lei Tong, Zheheng Jiang, Long Chen, Feixiang Zhou, Qianni Zhang, Xiangrong Zhang, Yaochu Jin, Huiyu Zhou
2021 arXiv   pre-print
A number of deep learning based methods have been applied to brain tumor segmentation and achieved promising results.  ...  In recent years, deep learning methods have shown promising performance in solving various computer vision problems, such as image classification, object detection and semantic segmentation.  ...  Paths with a small scale kernel can learn features from a smaller reception field known as local features.  ... 
arXiv:2007.09479v3 fatcat:vdbpwfdsorfudkvnvottexd7je

Multi-Gram CNN-Based Self-Attention Model for Relation Classification

Chunyun Zhang, Chaoran Cui, Sheng Gao, Xiushan Nie, Weiran Xu, Lu Yang, Xiaoming Xi, Yilong Yin
2019 IEEE Access  
The multi-gram conventional neural network attention model can learn the adaptive relational semantics of inputs based on the fact that a relation can be totally defined by the shortest dependency path  ...  To overcome the above-mentioned problems, many deep neural network-based methods have been proposed; however, these methods cannot effectively locate and utilize the relation trigger features.  ...  Deep neural networks have emerged recently and can learn robust underlying features automatically with promising results. Zeng et al.  ... 
doi:10.1109/access.2018.2888508 fatcat:q4zx657rkjewfmizt7z5jlubdy

Time Series Data Classification Based on Dual Path CNN-RNN Cascade Network

Chao Yang, Wenxiang Jiang, Zhongwen Guo
2019 IEEE Access  
We use a dual path CNN to achieve a multi-size receptive field for better feature extraction, then using RNN and the following fully-connected layers to learn the map between the given features and the  ...  Convolutional neural network (CNN) is a special kind of ANN that has been widely used in the area of image processing tasks as its ability for extracting spatial features.  ...  In our model, we use a dual path convolutional neural network to extract the features of the input data and use LSTM and the fully-connected layers to learn the map between the extracted features and the  ... 
doi:10.1109/access.2019.2949287 fatcat:ji7i4u3vzbebffqbek54yij6pu

A Modularized Architecture of Multi-Branch Convolutional Neural Network for Image Captioning

Shan He, Yuanyao Lu
2019 Electronics  
In this paper, we present an end-to-end model that takes deep convolutional neural network (CNN) as the encoder and recurrent neural network (RNN) as the decoder.  ...  Experiments are conducted on Flickr8k, Flickr30k and MSCOCO entities. Results demonstrate that our method achieves state of the art performances in terms of caption quality.  ...  LSTM The recurrent neural network is a mature technology in NLP and plays an important role in machine translation and speech recognition.  ... 
doi:10.3390/electronics8121417 fatcat:5avr5izj25d3ddw456lbyfd6em

CORENup: a combination of convolutional and recurrent deep neural networks for nucleosome positioning identification

Domenico Amato, Giosue' Lo Bosco, Riccardo Rizzo
2020 BMC Bioinformatics  
In this work, we propose CORENup, a deep learning model for nucleosome identification.  ...  The comparisons have been carried out using two groups of datasets, currently adopted by the best performing methods, and CORENup has shown top performance both in terms of classification metrics and elapsed  ...  This research is funded in part by MIUR Project of National Relevance 2017WR7SHH "Multicriteria Data Structures and Algorithms: from compressed to learned indexes, and beyond".  ... 
doi:10.1186/s12859-020-03627-x pmid:32938377 pmcid:PMC7493859 fatcat:p6gjortkrvgyhasbm6bmapoxpy

Learning Sparse Mixture of Experts for Visual Question Answering [article]

Vardaan Pahuja, Jie Fu, Christopher J. Pal
2019 arXiv   pre-print
In this project, we propose an efficient and modular neural architecture for the VQA task with focus on the CNN module.  ...  There has been a rapid progress in the task of Visual Question Answering with improved model architectures.  ...  Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering.  ... 
arXiv:1909.09192v1 fatcat:v2q5z377cndt7lfqcx2hibukdu

A Neurally-Inspired Hierarchical Prediction Network for Spatiotemporal Sequence Learning and Prediction [article]

Jielin Qiu, Ge Huang, Tai Sing Lee
2021 arXiv   pre-print
Within each level, the feed-forward path and the feedback path intersect in a recurrent gated circuit, instantiated in a LSTM module, to generate a prediction or explanation of the incoming signals.  ...  It contains a feed-forward path that computes and encodes spatiotemporal features of successive complexity and a feedback path for the successive levels to project their interpretations to the level below  ...  The network contains a feedforward path that is realized in a deep convolutional neural network (DCNN), a stack of Long Short Term Memory (LSTM) modules that link the feedforward path and the feedback  ... 
arXiv:1901.09002v2 fatcat:3yz3fenx2zaoje4gpi5ii7x4yi

Structure Regularized Bidirectional Recurrent Convolutional Neural Network for Relation Classification [article]

Ji Wen
2017 arXiv   pre-print
In this paper, we present a novel model, Structure Regularized Bidirectional Recurrent Convolutional Neural Network(SR-BRCNN), to classify the relation of two entities in a sentence, and the new dataset  ...  Some state-of-the-art systems concentrate on modeling the shortest dependency path (SDP) between two entities leveraging convolutional or recurrent neural networks.  ...  , we have seen a move towards deep architectures that are capable of learning relevant representations and features without extensive manual feature engineering or use of external resources.  ... 
arXiv:1711.02509v1 fatcat:wnehk2vemnd5vgo2h7gdwe23eu

Software defect prediction

Abdelrahman Ghunemi
In addition to powerful deep learning techniques. This paper used recurrent neural networks and convolutional neural networks to achieve an accuracy of 78%, and 91% respectively.  ...  Software defect prediction plays an important role in improving software quality and it helps to reduce cost, time, and resources.  ...  from its path. (3) Take each file and pass it through lexer and parser of javalang library. (4) mapping token vectors into integer vectors. (5) Save vectors in excel sheet to be fed to the deep learning  ... 
doi:10.6084/m9.figshare.14401304.v1 fatcat:2ngmsh337baf7n6xbvzybzaopu
« Previous Showing results 1 — 15 out of 3,375 results