1,076 Hits in 4.8 sec

Neural Speed Reading with Structural-Jump-LSTM [article]

Christian Hansen, Casper Hansen, Stephen Alstrup, Jakob Grue Simonsen, Christina Lioma
2019 arXiv   pre-print
We present Structural-Jump-LSTM: the first neural speed reading model to both skip and jump text during inference.  ...  A comprehensive experimental evaluation of our model against all five state-of-the-art neural reading models shows that Structural-Jump-LSTM achieves the best overall floating point operations (FLOP) reduction  ...  ://  ... 
arXiv:1904.00761v2 fatcat:imch75zsobfjlkcp7xania6m2e

Learning to Skim Text

Adams Wei Yu, Hongrae Lee, Quoc Le
2017 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)  
In our benchmarks on four different tasks, including number prediction, sentiment analysis, news article classification and automatic Q&A, our proposed model, a modified LSTM with jumping, is up to 6 times  ...  The underlying model is a recurrent network that learns how far to jump after reading a few words of the input text.  ...  As a result, the reading speed of LSTM-Jump is hardly affected by the length of sequence, but that of LSTM is linear with respect to length.  ... 
doi:10.18653/v1/p17-1172 dblp:conf/acl/YuLL17 fatcat:urnceqyrz5dapcpsu7md7qgkuy

Learning to Skim Text [article]

Adams Wei Yu, Hongrae Lee, Quoc V. Le
2017 arXiv   pre-print
In our benchmarks on four different tasks, including number prediction, sentiment analysis, news article classification and automatic Q\&A, our proposed model, a modified LSTM with jumping, is up to 6  ...  The underlying model is a recurrent network that learns how far to jump after reading a few words of the input text.  ...  As a result, the reading speed of LSTM-Jump is hardly affected by the length of sequence, but that of LSTM is linear with respect to length.  ... 
arXiv:1704.06877v2 fatcat:f7d7plkxuvhtpbvwxcceytly5u

A Survey on Dynamic Neural Networks for Natural Language Processing [article]

Canwen Xu, Julian McAuley
2022 arXiv   pre-print
Dynamic neural networks, as an emerging research direction, are capable of scaling up neural networks with sub-linear increases in computation and time by dynamically adjusting their computational path  ...  Dynamic neural networks could be a promising solution to the growing parameter numbers of pretrained language models, allowing both model pretraining with trillions of parameters and faster inference on  ...  Jump-LSTM [Hansen et al., 2019] hidden states stop reading; jump to next (,;) or (.!?)  ... 
arXiv:2202.07101v1 fatcat:c62x43swubhwzlfsw44cax7e5q

Sequential Modelling with Applications to Music Recommendation, Fact-Checking, and Speed Reading [article]

Christian Hansen
2021 arXiv   pre-print
read" text for efficient further classification.  ...  One example is systems that interact with users, log user actions and behaviour, and make recommendations of items of potential interest to users on the basis of their previous interactions.  ...  Neural Speed Reading with Structural-Jump-LSTM. In ICLR. [41].  ... 
arXiv:2109.06736v1 fatcat:xawmkvzhgng3vhhrs5xvwokqna

Neural Networks Model for Travel Time Prediction Based on ODTravel Time Matrix [article]

Ayobami E. Adewale, Amnir Hadachi
2020 arXiv   pre-print
In this study, two neural network models namely multi-layer(MLP) perceptron and long short-term model(LSTM) are developed for predicting link travel time of a busy route with input generated using Origin-Destination  ...  The experiment result showed that both models can make near-accurate predictions however, LSTM is more susceptible to noise as time step increases.  ...  Zhang et al in [10] developed a model with the fusion of 2-dimensional convolution neural network and LSTM.  ... 
arXiv:2004.04030v1 fatcat:tuigk77mnnhlbg6lvtxv3fzroi

Recent Trends in Deep Learning Based Open-Domain Textual Question Answering Systems

Zhen Huang, Shiyi Xu, Minghao Hu, Xinyi Wang, Jinyan Qiu, Yongquan Fu, Yuncai Zhao, Yuxing Peng, Changjian Wang
2020 IEEE Access  
Recent advancements in open-domain textual QA are mainly due to the significant developments of deep learning techniques, especially machine reading comprehension and neural-network-based information retrieval  ...  INDEX TERMS Open-domain textual question answering, deep learning, machine reading comprehension, information retrieval.  ...  [120] proposed the first speed reading model including both jump and skip actions. • Other speed reading applications: JUMPER [36] provided fast reading feedback for legal texts, Johansen and Socher  ... 
doi:10.1109/access.2020.2988903 fatcat:po4euxfronf3pob52qc2wcgrre

Structured Memory for Neural Turing Machines [article]

Wei Zhang, Yang Yu, Bowen Zhou
2015 arXiv   pre-print
In this paper, we propose several different structures of memory for NTM, and we proved in experiments that two of our proposed structured-memory NTMs could lead to better convergence, in term of speed  ...  Neural Turing Machines (NTM) contain memory component that simulates "working memory" in the brain to store and retrieve information to ease simple algorithms learning.  ...  Conclusion This paper discussed three new structured memory architectures for Neural Turing Machines, and showed that organizing memory blocks in a proper hierarchical manner could alleviate overfitting  ... 
arXiv:1510.03931v3 fatcat:zq4xyje56fc3pilhhswr74bz3a

StackReader: An RNN-Free Reading Comprehension Model

Yibo Jiang, Zhou Zhao
Many existing systems use RNN-based units for contextual modeling linked with some attention mechanisms.  ...  In this paper, however, we propose StackReader, an end-to-end neural network model, to solve this problem, without recurrent neural network (RNN) units and its variants.  ...  It starts with a selfattention modeling layer and then it jumps from interactive attention layer to self-attention layer iteratively.  ... 
doi:10.1609/aaai.v32i1.12169 fatcat:v6d5btkw7fharekfuk3bi4vqzq

Automatic Detection of Atrial Fibrillation Based on CNN-LSTM and Shortcut Connection

Yongjie Ping, Chao Chen, Lu Wu, Yinglong Wang, Minglei Shu
2020 Healthcare  
In this paper, a combination of an 8-layer convolutional neural network (CNN) with a shortcut connection and 1-layer long short-term memory (LSTM), named 8CSL, was proposed for the Electrocardiogram (ECG  ...  Compared with recurrent neural networks (RNN) and multi-scale convolution neural networks (MCNN), not only can 8CSL extract features skillfully, but also deal with long-term dependency between data.  ...  Long Short-Term Memory (LSTM) Structure The working mechanism of LSTM is the continuously updated memory c n .  ... 
doi:10.3390/healthcare8020139 pmid:32443926 pmcid:PMC7348856 fatcat:u5qqewkjyjhwhgmexb7ee4ysiq

Regulated LSTM Artificial Neural Networks for Option Risks

David Liu, An Wei
2022 FinTech  
This research aims to study the pricing risks of options by using improved LSTM artificial neural network models and make direct comparisons with the Black–Scholes option pricing model based upon the option  ...  We study an LSTM model, a mathematical option pricing model (BS model), and an improved artificial neural network model—the regulated LSTM model.  ...  [16] established that by applying a moving average time strategy to a portfolio structured in accordance with a book-to-market ratio, a higher rate of return can be generated than with a buy-and-hold  ... 
doi:10.3390/fintech1020014 fatcat:d52a2aoqmndejp7wyq7kughc6a

Compositional Generalization via Neural-Symbolic Stack Machines [article]

Xinyun Chen, Chen Liang, Adams Wei Yu, Dawn Song, Denny Zhou
2020 arXiv   pre-print
NeSS combines the expressive power of neural sequence models with the recursion supported by the symbolic stack machine.  ...  It contains a neural network to generate traces, which are then executed by a symbolic stack machine enhanced with sequence manipulation operations.  ...  Specifically, Stack LSTM, Queue LSTM, and DeQue LSTM are designed in [18] , where they augment an LSTM with a differentiable data structure.  ... 
arXiv:2008.06662v2 fatcat:5hcytak7zjdj5eotga3yzwgpyi

An LSTM with Differential Structure and Its Application in Action Recognition

Weifeng Chen, Fei Zheng, Shanping Gao, Kai Hu
2022 Mathematical Problems in Engineering  
Thus, an improved LSTM structure with an input differential characteristic module is proposed.  ...  technology to reflect the change of speed.  ...  , jumping, running, standing, hopping, walking, waving1, and waving2).  ... 
doi:10.1155/2022/7316396 doaj:610cf493a1de4a6d97bf9d4b7ff8ab26 fatcat:o3didkxcezgw7kzoontquua5qe

A novel DL approach to PE malware detection: exploring Glove vectorization, MCC_RCNN and feature fusion [article]

Yuzhou Lin
2021 arXiv   pre-print
We implement a neural network model called MCC_RCNN (Malware Detection and Recurrent Convolutional Neural Network), comprising of the combination with CNN and RNN.  ...  With the numerical results generated from several comparative experiments towards evaluating the Glove-based vectoriza-tion, MCC_RCNN-based classification methodology and feature fusion stages, our proposed  ...  LSTM can handle with sequences but lack of the ability to processing long-length data inputs in a fast speed. Compared with it, Dauphin et al.  ... 
arXiv:2101.08969v3 fatcat:7hj4xy4mufeznhysdcekcq23vy

Estimation of Mechanical Power Output Employing Deep Learning on Inertial Measurement Data in Roller Ski Skating

Md Zia Uddin, Trine M. Seeberg, Jan Kocbach, Anders E. Liverud, Victor Gonzalez, Øyvind Sandbakk, Frédéric Meyer
2021 Sensors  
Thirteen athletes conducted roller ski skating trials on a treadmill with varying incline and speed.  ...  In this paper, we aim to estimate mechanical power output by employing a time-sequential information-based deep Long Short-Term Memory (LSTM) neural network from multiple inertial measurement units (IMUs  ...  A basic structure of the LSTM-based RNN. Each LSTM memory unit consists of three important gates: input, forget and the output gate.  ... 
doi:10.3390/s21196500 pmid:34640819 fatcat:l5t4nnyewjepzl5rfr3zi6fiva
« Previous Showing results 1 — 15 out of 1,076 results