Filters








2,488 Hits in 7.0 sec

Pretrained Language Models for Sequential Sentence Classification

Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Dan Weld
2019 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)  
Our approach achieves state-of-the-art results on four datasets, including a new dataset of structured scientific abstracts.  ...  Specifically, we construct a joint sentence representation that allows BERT Transformer layers to directly utilize contextual information from all words in all sentences.  ...  Our model is a single model with only one added feature.  ... 
doi:10.18653/v1/d19-1383 dblp:conf/emnlp/CohanBKDW19 fatcat:r6c3grxwqva73jb355tcnackq4

A Keyword Detection and Context Filtering Method for Document Level Relation Extraction

Hailan Kuang, Haoran Chen, Xiaolin Ma, Xinhua Liu
2022 Applied Sciences  
In addition, a Self-Attention Memory (SAM) module in ConvLSTM is introduced to process the document context and capture keyword features.  ...  This paper proposes a keyword detection and context filtering method based on the Self-Attention mechanism for document-level RE.  ...  In addition, considering that the BERT model also comes with a multi-headed attention mechanism, we directly remove the LSTM layer and test the effect of the BERT+classifier.  ... 
doi:10.3390/app12031599 fatcat:epp5tyab4ngmbk6w25mikuwnly

A New Entity Extraction Method Based on Machine Reading Comprehension [article]

Xiaobo Jiang, Kun He, Jiajun He, Guangyu Yan
2021 arXiv   pre-print
Experiments have proved that MRC-I2DP represents an overall state-of-the-art model in 7 from the scientific and public domains, achieving a performance improvement of up to compared to the model model  ...  Entity extraction is a key technology for obtaining information from massive texts in natural language processing.  ...  The overall architecture of our MRC-I2DP, as shown in Fig. 2 , consists of three main modules.  The Pre-trained module: conducts a feature extraction of the input after BPE tokenization by deep self-attention  ... 
arXiv:2108.06444v2 fatcat:lwdbzvtyfraotid6qdfjerawqi

An Entity Relation Extraction Method Based on Dynamic Context and Multi-Feature Fusion

Xiaolin Ma, Kaiqi Wu, Hailan Kuang, Xinhua Liu
2022 Applied Sciences  
The context area is picked dynamically with the help of threshold in feature selecting layer of the model.  ...  To address the problem, we propose a span-based joint extraction method based on dynamic context and multi-feature fusion (SPERT-DC).  ...  . • This paper proposes a Bi-LSTM combined with self-attention (Bi-LSTM_ATT).  ... 
doi:10.3390/app12031532 fatcat:yea4bfoa4raqbdrj77ch25vchm

Machine learning based natural language processing of radiology reports in orthopaedic trauma

A.W. Olthof, P. Shouche, E.M. Fennema, F.F.A. IJpma, R.H.C. Koolstra, V.M.A. Stirler, P.M.A. van Ooijen, L.J. Cornelissen
2021 Computer Methods and Programs in Biomedicine  
BERT has a very deep model architecture, based on self-attention layers [38] , allowing the model to learn relationships between individual words as well as sentences as a whole.  ...  With 7 7 PubMed records in 2019 and 52 in 2020 the scientific literature on BERT in healthcare is limited but expanding.  ... 
doi:10.1016/j.cmpb.2021.106304 fatcat:lbsobflqrncjzjxsvmlr5rnbvm

Meta-Analysis of Cross-Language Plagiarism and Self-Plagiarism Detection Methods for Russian-English Language Pair

Alina Tlitova, Alexander Toschev, Max Talanov, Vitaliy Kurnosov
2020 Frontiers in Computer Science  
Citation: Tlitova A, Toschev A, Talanov M and Kurnosov V (2020) Meta-Analysis of Cross-Language Plagiarism and Self-Plagiarism Detection Methods for Russian-English Language Pair. Front. Comput.  ...  Along with the problems of authorship, paid research, fabrication of results, plagiarism, and self-plagiarism are among the most common violations.  ...  The authors of Zubarev and Sochenkov (2019) , therefore proposed a classifier with a reduced space for features for effective filtering: only with sentence embeddings and word substitution measures.  ... 
doi:10.3389/fcomp.2020.523053 fatcat:2eqhh655mbb3jedkai74igfk7q

Transformer-F: A Transformer network with effective methods for learning universal sentence representation [article]

Yu Shi
2021 arXiv   pre-print
We calculated the attention score by multiplying the part-of-speech weight vector with the correlation coefficient, which helps extract the words with more practical meaning.  ...  Specifically, we obtain a 5.28% relative improvement over the vanilla Transformer on the simple tasks.  ...  The authors declare that there is no conflict of interest regarding the publication of this article.  ... 
arXiv:2107.00653v1 fatcat:kj2fsnokzfbblokzh4kshep57y

Pre-training Methods in Information Retrieval [article]

Yixing Fan, Xiaohui Xie, Yinqiong Cai, Jia Chen, Xinyu Ma, Xiangsheng Li, Ruqing Zhang, Jiafeng Guo
2022 arXiv   pre-print
The core of information retrieval (IR) is to identify relevant information from large-scale resources and return it as a ranked list to respond to the user's information need.  ...  Considering the rapid progress of this direction, this survey aims to provide a systematic review of pre-training methods in IR.  ...  Acknowledgements References Pre-training Methods in Information Retrieval Acknowledgements  ... 
arXiv:2111.13853v3 fatcat:pilemnpphrgv5ksaktvctqdi4y

Research on Fine-Grained Classification of Rumors in Public Crisis ——Take the COVID-19 incident as an example

Shuaipu Chen, A. Ghadouani, F. Wu
2020 E3S Web of Conferences  
[Method / Process] Based on the rumor data of several mainstream rumor refuting platforms, the pre-training model of BERT was used to fine-tuning in the context of COVID-19 events to obtain the feature  ...  vector representation of the rumor sentence level to achieve fine-grained classification, and a comparative experiment was conducted with the TextCNN and TextRNN models.  ...  The Encoder side and Decoder side are based on the self-attention mechanism for feature selection, so the working principle of the two is basically the same.  ... 
doi:10.1051/e3sconf/202017902027 fatcat:3n6nv7tiyncvffhbvj6s3to5ne

Improving Scholarly Knowledge Representation: Evaluating BERT-based Models for Scientific Relation Classification [article]

Ming Jiang, Jennifer D'Souza, Sören Auer, J. Stephen Downie
2020 arXiv   pre-print
With the rapid growth of research publications, there is a vast amount of scholarly knowledge that needs to be organized in digital libraries.  ...  Within such graph-based pipelines, inferring relation types between related scientific concepts is a crucial step.  ...  In this case, however, the core BERT architecture's self-attention mechanism is modified to efficiently consider the representations of the relative positions of scientific terms [20, 26] .  ... 
arXiv:2004.06153v2 fatcat:sr44tjxh4fdz7bhxqbfps6uu6e

Authorship Attribution of Social Media and Literary Russian-Language Texts Using Machine Learning Methods and Feature Selection

Anastasia Fedotova, Aleksandr Romanov, Anna Kurtukova, Alexander Shelupanov
2021 Future Internet  
A particular experiment was devoted to the selection of informative features using genetic algorithms (GA) and evaluation of the classifier trained on the optimal feature space.  ...  Using fastText or a combination of support vector machine (SVM) with GA reduced the time costs by half in comparison with deep NNs with comparable accuracy.  ...  It combines the advantages of CNN with Self-attention and Transformer that discussed in the previous work [1] , and at the same time, BERT allows for higher accuracy in related tasks [49] .  ... 
doi:10.3390/fi14010004 fatcat:u2aqr6jurrabjlr6bizziibxhi

Survey of Generative Methods for Social Media Analysis [article]

Stan Matwin, Aristides Milios, Paweł Prałat, Amilcar Soares, François Théberge
2021 arXiv   pre-print
This survey draws a broad-stroke, panoramic picture of the State of the Art (SoTA) of the research in generative methods for the analysis of social media data.  ...  It fills a void, as the existing survey articles are either much narrower in their scope or are dated.  ...  The success of BERT with regards to ABSA has been analyzed through the visualization of the fine-tuned BERT model self-attention heads, to see what features are being used in this classification task,  ... 
arXiv:2112.07041v1 fatcat:xgmduwctpbddfo67y6ack5s2um

Sentiment Analysis Method of Network Text Based on Improved AT-BiGRU Model

Xinxin Lu, Hong Zhang, Wenzheng Bao
2021 Scientific Programming  
Secondly, pad_sequences are used to fill in the input layer with a fixed length, the two-way gated recurrent network is used to extract information, and the attention mechanism is used to highlight the  ...  text sentiment analysis method based on the improved AT-BiGRU model.  ...  Acknowledgments is study was supported by the 2021 Key Development Plan Project of the Science and Technology Department of Henan Province (no. 212102210400).  ... 
doi:10.1155/2021/6669664 fatcat:7c72t7qwfzbm3gqwyl4nwoxejy

A Method about Building Deep Knowledge Graph for the Plant Insect Pest and Disease (DKG-PIPD)

Yingying Liu
2021 IEEE Access  
The experimental contrast results with other classical benchmark methods demonstrated the effectiveness of the proposed method.  ...  Furthermore, DKG-PIPD performed joint extraction about the entity and the relationship in unstructured knowledge in a corpus tagging method that is suitable for domain data.  ...  , the segment vector and the position vector; {T1, T2 ,…, Tn } is the target of BERT, which is a sequence vector with rich semantic features obtained after feature extraction by the bi-directional converter  ... 
doi:10.1109/access.2021.3116467 fatcat:vyoluv7ln5ebnlrm3ogut32eoa

Pretrained Language Models for Sequential Sentence Classification [article]

Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Daniel S. Weld
2019 arXiv   pre-print
Our approach achieves state-of-the-art results on four datasets, including a new dataset of structured scientific abstracts.  ...  Specifically, we construct a joint sentence representation that allows BERT Transformer layers to directly utilize contextual information from all words in all sentences.  ...  Our model is a single model with only one added feature.  ... 
arXiv:1909.04054v2 fatcat:m2rldlbdjrhyrhebp4i3nrhphy
« Previous Showing results 1 — 15 out of 2,488 results