Filters








64 Hits in 10.3 sec

A Battle of Network Structures: An Empirical Study of CNN, Transformer, and MLP [article]

Yucheng Zhao, Guangting Wang, Chuanxin Tang, Chong Luo, Wenjun Zeng, Zheng-Jun Zha
2021 arXiv   pre-print
In this paper, we conduct empirical studies on these DNN structures and try to understand their respective pros and cons.  ...  Recently, Transformer and multi-layer perceptron (MLP)-based models, such as Vision Transformer and MLP-Mixer, started to lead new trends as they showed promising results in the ImageNet classification  ...  Empirical Studies on Mixing Blocks In this section, we design a series of controlled experiments to compare the three network structures.  ... 
arXiv:2108.13002v2 fatcat:isudzxxth5awtmnx2p5k52pwgu

Hformer: Hybrid CNN-Transformer for Fringe Order Prediction in Phase Unwrapping of Fringe Projection [article]

Xinjun Zhu, Zhiqiang Han, Mengkai Yuan, Qinghua Guo, Hongyi Wang
2021 arXiv   pre-print
The proposed model has a hybrid CNN-Transformer architecture that is mainly composed of backbone, encoder and decoder to take advantage of both CNN and Transformer.  ...  Convolutional Neural Network (CNN) models.  ...  Zheng, “A battle of network structures: An empirical study of CNN, Transformer, and MLP,” arXiv preprint arXiv:2108.13002 (2020). 20. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X.  ... 
arXiv:2112.06759v1 fatcat:tdcd3kygajfknf4pd3kpph4viy

End-to-end Graph-based TAG Parsing with Neural Networks [article]

Jungo Kasai and Robert Frank and Pauli Xu and William Merrill and Owen Rambow
2018 arXiv   pre-print
We present a graph-based Tree Adjoining Grammar (TAG) parser that uses BiLSTMs, highway connections, and character-level CNNs.  ...  This provides further support for the claim that TAG is a viable formalism for problems that require rich structural analysis of sentences.  ...  Empirical studies have shown that a transitionbased parser and a graph-based parser yield similar overall performance across languages (Mc-Donald and Nivre, 2011) , but the two strands of data-driven  ... 
arXiv:1804.06610v3 fatcat:eoy6lpducbcxhksg555qmqwlte

End-to-End Graph-Based TAG Parsing with Neural Networks

Jungo Kasai, Robert Frank, Pauli Xu, William Merrill, Owen Rambow
2018 Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)  
We present a graph-based Tree Adjoining Grammar (TAG) parser that uses BiL-STMs, highway connections, and characterlevel CNNs.  ...  This provides further support for the claim that TAG is a viable formalism for problems that require rich structural analysis of sentences.  ...  Empirical studies have shown that a transitionbased parser and a graph-based parser yield similar overall performance across languages (Mc-Donald and Nivre, 2011) , but the two strands of data-driven  ... 
doi:10.18653/v1/n18-1107 dblp:conf/naacl/KasaiFXMR18 fatcat:sntxt3ijgjgene7xacibzril7i

Deep Learning in Mobile and Wireless Networking: A Survey [article]

Chaoyun Zhang, Paul Patras, Hamed Haddadi
2019 arXiv   pre-print
In this paper we bridge the gap between deep learning and mobile and wireless networking research, by presenting a comprehensive survey of the crossovers between the two areas.  ...  Subsequently, we provide an encyclopedic review of mobile and wireless networking research based on deep learning, which we categorize by different domains.  ...  Fig. 1 : 1 Diagramatic view of the organization of this survey. Fig. 6 : 6 Typical structure and operation principles of MLP, RBM, AE, CNN, RNN, LSTM, GAN, and DRL.  ... 
arXiv:1803.04311v3 fatcat:awuvyviarvbr5kd5ilqndpfsde

Neural Network–Based Financial Volatility Forecasting: A Systematic Review

Wenbo Ge, Pooia Lalbakhsh, Leigh Isai, Artem Lenskiy, Hanna Suominen
2023 ACM Computing Surveys  
A snapshot of state-of-the-art neural network–based financial volatility forecasting was generated by examining 35 studies, published after 2015.  ...  Finally, adequate background was provided to serve as an introduction to the field of neural network volatility forecasting.  ...  Further categorisation of the NN model was made into three broad classes: "MLP, " "RNN, " and "CNN. " The presence of an MLP in conjunction with the RNN or CNN classes did not justify including it as an  ... 
doi:10.1145/3483596 fatcat:gsyyg72xjbaolb2vpdstenjt5a

Multi-head Cascaded Swin Transformers with Attention to k-space Sampling Pattern for Accelerated MRI Reconstruction [article]

Mevan Ekanayake, Kamlesh Pawar, Mehrtash Harandi, Gary Egan, Zhaolin Chen
2022 arXiv   pre-print
The existing contributions mostly provide CNN-transformer hybrid solutions and rarely leverage the physics of MRI.  ...  Global correlations are widely seen in human anatomical structures due to similarity across tissues and bones.  ...  Acknowledgments The authors are grateful for support from Australian Research Council Linkage grant LP170100494 and Australian Research Council Discovery grant DP210101863.  ... 
arXiv:2207.08412v1 fatcat:ewu3rsg2tvck7oybmcua4smjje

Graph Convolutional Reinforcement Learning [article]

Jiechuan Jiang, Chen Dun, Tiejun Huang, Zongqing Lu
2020 arXiv   pre-print
Empirically, we show that our method substantially outperforms existing methods in a variety of cooperative scenarios.  ...  To tackle these difficulties, we propose graph convolutional reinforcement learning, where graph convolution adapts to the dynamics of the underlying graph of the multi-agent environment, and relation  ...  ACKNOWLEDGMENTS This work was supported in part by NSF China under grant 61872009, Huawei Noah's Ark Lab, and Peng Cheng Lab.  ... 
arXiv:1810.09202v5 fatcat:lzyggbsbxva7rcgim3np6gru44

Applications of Artificial Intelligence in Battling Against Covid-19: A Literature Review

Mohammad-H. Tayarani-N.
2020 Chaos, Solitons & Fractals  
Here, we perform an overview on the applications of AI in a variety of fields including diagnosis of the disease via different types of tests and symptoms, monitoring patients, identifying severity of  ...  To manage the problems, many research in a variety of area of science have started studying the issue.  ...  An empirical top-down modeling algorithm is proposed in [476] , which uses a combination of epidemiological, statistical and neural network applications.  ... 
doi:10.1016/j.chaos.2020.110338 pmid:33041533 pmcid:PMC7532790 fatcat:gl3i37hag5gflajsa7fh6khvva

A Cost-Aware DNN-Based FDI Technology for Solenoid Pumps

Suju Kim, Ugochukwu Ejike Akpudo, Jang-Wook Hur
2021 Electronics  
This study employs a deep neural network (DNN) for fault diagnosis after a correlation-based selection of discriminative spectral and transient features.  ...  , etc.) on different network architectures.The results reveal the high accuracy of a three-layer DNN with ReLU activation function, with a test accuracy of 99.23% and a minimal false alarm rate on a case  ...  Conflicts of Interest: The authors declare no conflict of interest.  ... 
doi:10.3390/electronics10192323 fatcat:fueotqzl6zhlniucu54ov5kb2i

Towards Autoencoding Variational Inference for Aspect-based Opinion Summary [article]

Tai Hoang, Huy Le, Tho Quan
2019 arXiv   pre-print
In this research, we study an alternative approach for AOS problem, based on Autoencoding Variational Inference (AVI).  ...  Along this direction, the LDA-based model is considered as a notably suitable approach, since this model offers both topic modeling and sentiment classification.  ...  Just for example, in this work we employed MLP and the state-of-the-art CNN deep networks for classification.  ... 
arXiv:1902.02507v2 fatcat:arwqphelbnfzpba26vpcc4dfcq

Stock Price Movement Prediction Based on a Deep Factorization Machine and the Attention Mechanism

Xiaodong Zhang, Suhui Liu, Xin Zheng
2021 Mathematics  
In this paper, we constructed a convolutional neural network model based on a deep factorization machine and attention mechanism (FA-CNN) to improve the prediction accuracy of stock price movement via  ...  The prediction of stock price movement is a popular area of research in academic and industrial fields due to the dynamic, highly sensitive, nonlinear and chaotic nature of stock prices.  ...  [46] proposed an empirical mode decomposition and factorization machine based neural network (EMD2FNN) model to predict the value of several stock indices.  ... 
doi:10.3390/math9080800 fatcat:lphqvfrzf5gyhdytupttbnxpdi

HIN: Hierarchical Inference Network for Document-Level Relation Extraction [article]

Hengzhu Tang, Yanan Cao, Zhenyu Zhang, Jiangxia Cao, Fang Fang, Shi Wang, Pengfei Yin
2020 arXiv   pre-print
In this paper, we propose a Hierarchical Inference Network (HIN) to make full use of the abundant information from entity level, sentence level and document level.  ...  Translation constraint and bilinear transformation are applied to target entity pair in multiple subspaces to get entity-level inference information.  ...  Ablation Study To study the contribution of each component in HIN-BERT, we run an ablation study on DocRED dev set (see Table 2 ).  ... 
arXiv:2003.12754v1 fatcat:nv5gbv5zvzdyzob3tx6k2rxbf4

HIN: Hierarchical Inference Network for Document-Level Relation Extraction [chapter]

Hengzhu Tang, Yanan Cao, Zhenyu Zhang, Jiangxia Cao, Fang Fang, Shi Wang, Pengfei Yin
2020 Lecture Notes in Computer Science  
In this paper, we propose a Hierarchical Inference Network (HIN) to make full use of the abundant information from entity level, sentence level and document level.  ...  Translation constraint and bilinear transformation are applied to target entity pair in multiple subspaces to get entity-level inference information.  ...  Ablation Study. To study the contribution of each component in HIN-BERT, we run an ablation study on DocRED dev set (see Table 2 ).  ... 
doi:10.1007/978-3-030-47426-3_16 fatcat:mrgidhlz6vdivi57b6mhvlp7zm

Rethinking movie genre classification with fine-grained semantic clustering [article]

Edward Fish, Jon Weinbren, Andrew Gilbert
2021 arXiv   pre-print
By leveraging pre-trained 'expert' networks, we learn the influence of different combinations of modes for multi-label genre classification.  ...  This leads to a more 'fine-grained' and detailed clustering, based on semantic similarities while still retaining some genre information.  ...  First the sequence embeddings x are summed over a trailer and then projected via an MLP k(.) to produce a logits embedding.  ... 
arXiv:2012.02639v3 fatcat:sufthmjshjdtpdk7ijgitlkhqm
« Previous Showing results 1 — 15 out of 64 results