Filters








200 Hits in 4.6 sec

SciFive: a text-to-text transformer model for biomedical literature [article]

Long N. Phan, James T. Anibal, Hieu Tran, Shaurya Chanana, Erol Bahadroglu, Alec Peltekian, Grégoire Altan-Bonnet
2021 arXiv   pre-print
BERT, BioBERT, Base T5) on tasks in named entity relation, relation extraction, natural language inference, and question-answering.  ...  In this report, we introduce SciFive, a domain-specific T5 model that has been pre-trained on large biomedical corpora. Our model outperforms the current SOTA methods (i.e.  ...  In one example, we leverage this type of training for the Named-entity recognition task.  ... 
arXiv:2106.03598v1 fatcat:vce5ki2sone3nab4jz2gp3moci

BioBERT: a pre-trained biomedical language representation model for biomedical text mining [article]

Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, Jaewoo Kang
2019 arXiv   pre-print
Based on the BERT architecture, BioBERT effectively transfers the knowledge from a large amount of biomedical texts to biomedical text mining models with minimal task-specific architecture modifications  ...  entity recognition (0.51% absolute improvement), biomedical relation extraction (3.49% absolute improvement), and biomedical question answering (9.61% absolute improvement).  ...  With minimum architecture modification, BioBERT outperforms the current state-of-the-art models in biomedical named entity recognition by 0.51 F1 score, biomedical relation extraction by 3.49 F1 score  ... 
arXiv:1901.08746v3 fatcat:gchgrvjqqbb2dlfujbitgfiqee

Analyzing the Effect of Multi-task Learning for Biomedical Named Entity Recognition [article]

Arda Akdemir, Tetsuo Shibuya
2020 arXiv   pre-print
Besides, we propose combining transfer learning and multi-task learning to improve the performance of biomedical named entity recognition systems, which is not applied before to the best of our knowledge  ...  Developing high-performing systems for detecting biomedical named entities has major implications.  ...  In this study, we analyze the effect of multi-task learning for biomedical named entity recognition.  ... 
arXiv:2011.00425v1 fatcat:5tpxymmqfvanjm2pxmsscpphde

On Biomedical Named Entity Recognition: Experiments in Interlingual Transfer for Clinical and Social Media Texts [chapter]

Zulfat Miftahutdinov, Ilseyar Alimova, Elena Tutubalina
2020 Lecture Notes in Computer Science  
Although deep neural networks yield state-of-the-art performance in biomedical named entity recognition (bioNER), much research shares one limitation: models are usually trained and evaluated on English  ...  We demonstrate that pretraining on data with one or both types of transfer can be effective.  ...  We thank Sergey Nikolenko for helpful discussions. This research was supported by the Russian Science Foundation grant # 18-11-00284.  ... 
doi:10.1007/978-3-030-45442-5_35 fatcat:q6kplymurfbnbimgf5iejkdevi

PharmKE: Knowledge Extraction Platform for Pharmaceutical Texts using Transfer Learning [article]

Nasi Jofche, Kostadin Mishev, Riste Stojanov, Milos Jovanovik, Dimitar Trajanov
2021 arXiv   pre-print
Additionally, the PharmKE platform integrates the results obtained from named entity recognition tasks to resolve co-references of entities and analyze the semantic relations in every sentence, thus setting  ...  It performs text classification using state-of-the-art transfer learning models, and thoroughly integrates the results obtained through a proposed methodology.  ...  Hakala and Pyysalo [25] present an approach based on Conditional Random Fields (CRF) and multilingual BERT for biomedical named entity recognition on content in Spanish.  ... 
arXiv:2102.13139v1 fatcat:etxl7wybtzec7chi62cytui754

BioMegatron: Larger Biomedical Domain Language Model [article]

Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, Raghav Mani
2020 arXiv   pre-print
We demonstrate noticeable improvements over the previous state-of-the-art (SOTA) on standard biomedical NLP benchmarks of named entity recognition, relation extraction, and question answering.  ...  There has been an influx of biomedical domain-specific language models, showing language models pre-trained on biomedical text perform better on biomedical domain benchmarks than those trained on general  ...  Acknowledgement The authors would like to thank Sun Kim at NIH/NCBI (now at Amazon Alexa AI) for helpful discussions and suggestions.  ... 
arXiv:2010.06060v2 fatcat:2dlnr27zm5fv7hpbtlw7ka4fv4

Unsupervised Pre-training for Biomedical Question Answering [article]

Vaishnavi Kommaraju, Karthick Gunasekaran, Kun Li, Trapit Bansal, Andrew McCallum, Ivana Williams, Ana-Maria Istrate
2020 arXiv   pre-print
We explore the suitability of unsupervised representation learning methods on biomedical text -- BioBERT, SciBERT, and BioSentVec -- for biomedical question answering.  ...  Our pre-training method consists of corrupting a given context by randomly replacing some mention of a biomedical entity with a random entity mention and then querying the model with the correct entity  ...  Acknowledgements This work was supported in part by the UMass Amherst Center for Data Science and the Center for Intelligent Information Retrieval, in part by the Chan Zuckerberg Initiative, and in part  ... 
arXiv:2009.12952v1 fatcat:lwjccrybcrcjpon3tnoxavazsa

BioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity Recognition [article]

Usman Naseem, Matloob Khushi, Vinay Reddy, Sakthivel Rajendran, Imran Razzak, Jinman Kim
2020 arXiv   pre-print
In recent years, with the growing amount of biomedical documents, coupled with advancement in natural language processing algorithms, the research on biomedical named entity recognition (BioNER) has increased  ...  In our experiments, BioALBERT outperformed comparative SOTA BioNER models on eight biomedical NER benchmark datasets with four different entity types.  ...  In this study, we presented BioALBERT, which is a pretrained language model for biomedical named entity recognition.  ... 
arXiv:2009.09223v1 fatcat:odojdp7bjzc6biupgmjyvv2d74

Improving the recall of biomedical named entity recognition with label re-correction and knowledge distillation

Huiwei Zhou, Zhe Liu, Chengkun Lang, Yibin Xu, Yingyu Lin, Junjie Hou
2021 BMC Bioinformatics  
Background Biomedical named entity recognition is one of the most essential tasks in biomedical information extraction.  ...  Methods To remedy the above issue, we propose a novel Biomedical Named Entity Recognition (BioNER) framework with label re-correction and knowledge distillation strategies, which could not only create  ...  Acknowledgements We would like to thank the editors and all anonymous reviewers for valuable suggestions and constructive comments Authors' contributions HZ and ZL designed the study.  ... 
doi:10.1186/s12859-021-04200-w pmid:34078270 fatcat:2zbg3vqqvjcmjoqe24nrhzhdrq

Knowledge Guided Named Entity Recognition for BioMedical Text [article]

Pratyay Banerjee, Kuntal Kumar Pal, Murthy Devarakonda, Chitta Baral
2020 arXiv   pre-print
We provide different knowledge contexts, such as, entity types, questions, definitions and examples along with the text and train on a combined dataset of 18 biomedical corpora.  ...  In this work, we formulate the NER task as a multi-answer knowledge guided QA task (KGQA) which helps to predict entities only by assigning B, I and O tags without associating entity types with the tags  ...  We train the model with following hyperparameters, learning rates in the range [1e-6,5e-5], batch sizes of [16,32,48,64], linear weight-decay in range [0.001,0.1] and warm-up steps in range of [100,1000  ... 
arXiv:1911.03869v4 fatcat:j2avaf43yvamfgrgd66jt3xw3m

MT-BioNER: Multi-task Learning for Biomedical Named Entity Recognition using Deep Bidirectional Transformers [article]

Muhammad Raza Khan, Morteza Ziyadi, Mohamed AbdelHady
2020 arXiv   pre-print
The support of a new domain includes the design and development of a number of NLU components for domain classification, intents classification and slots tagging (including named entity recognition).  ...  The experimental results on the biomedical domain have shown that the proposed approach outperforms the previous state-of-the-art systems for slot tagging on the different benchmark biomedical datasets  ...  In contrast, our model MT-BioNER is optimized for biomedical named entity recognition using BioBERT as the shared layers and the different data sets in the task-specific layers.  ... 
arXiv:2001.08904v1 fatcat:ox3emhzakrgfzljczjlpuyjnhi

External features enriched model for biomedical question answering

Gezheng Xu, Wenge Rong, Yanmeng Wang, Yuanxin Ouyang, Zhang Xiong
2021 BMC Bioinformatics  
Results Inspired by the importance of syntactic and lexical features in the biomedical corpus, we proposed a new framework to extract external features, such as part-of-speech and named-entity recognition  ...  However, considering the lexical characteristics of biomedical corpus and its small scale dataset, there is still much improvement room for biomedical QA tasks.  ...  by the biomedical NER that merely focuses on a specific biomedical category named entity's recognition such as disease entity or gene entity; besides, for a biomedical question like "What is the genetic  ... 
doi:10.1186/s12859-021-04176-7 pmid:34039273 fatcat:shdrzlzntbhapnmcsd2q5hwr7u

Coronavirus Knowledge Graph: A Case Study [article]

Chongyan Chen, Islam Akef Ebeid, Yi Bu, Ying Ding
2020 arXiv   pre-print
COVID-19 related experts and bio-entities.  ...  One of the essential Knowledge Discovery tools that could help the biomedical research community understand and eventually find a cure for COVID-19 are Knowledge Graphs.  ...  Jaewoo Kange's DMIS Lab team for pretraining BioBERT, Vinay Locharulu for suggestion and support, Prof. Jian Xu for providing PubMed Knowledge Graph, and Yifei Wu for conducting entity normalization.  ... 
arXiv:2007.10287v1 fatcat:rzgsevv3jrhvrh74s34qyjof4q

DeepVar: An End-to-End Deep Learning Approach for Genomic Variant Recognition in Biomedical Literature

Chaoran Cheng, Fei Tan, Zhi Wei
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
We consider the problem of Named Entity Recognition (NER) on biomedical scientific literature, and more specifically the genomic variants recognition in this work.  ...  In this work, we present the first successful end-to-end deep learning approach to bridge the gap between generic NER algorithms and low-resource applications through genomic variants recognition.  ...  The Biomedical Named Entity Recognition (BioNER) tasks focus on extracting biomedical domain entities such as cell lines, diseases, genes, and proteins.  ... 
doi:10.1609/aaai.v34i01.5399 fatcat:crrsiaoinjhthfw7vba27rug6q

Bleeding Entity Recognition in Electronic Health Records: A Comprehensive Analysis of End-to-End Systems

Avijit Mitra, Bhanu Pratap Singh Rawat, David McManus, Alok Kapoor, Hong Yu
2021 AMIA Annual Symposium Proceedings  
Using our expert-annotated 1,079 de-identified EHR notes, we evaluated state-of-the-art NLP models such as biLSTM-CRF with language modeling, and different BERT variants for six entity types.  ...  On our dataset, the biLSTM-CRF surpassed other models resulting in a macro F1-score of 0.75 whereas the performance difference is negligible for sentence and document-level predictions with the best macro  ...  Additionally, we would like to thank our anonymous reviewers for their constructive comments on this work.  ... 
pmid:33936461 pmcid:PMC8075442 fatcat:jwhasrhkwrfwzdqzrd6ceulnou
« Previous Showing results 1 — 15 out of 200 results