Filters








334,371 Hits in 10.2 sec

Clinical Relation Extraction Using Transformer-based Models [article]

Xi Yang, Zehao Yu, Yi Guo, Jiang Bian, Yonghui Wu
2021 arXiv   pre-print
The goal of this study is to systematically explore three widely used transformer-based models (i.e., BERT, RoBERTa, and XLNet) for clinical relation extraction and develop an open-source package with  ...  clinical pre-trained transformer-based models to facilitate information extraction in the clinical domain.  ...  We gratefully acknowledge the support of NVIDIA Corporation and NVIDIA AI Technology Center with the donation of the GPUs and the computing resources used for this research.  ... 
arXiv:2107.08957v2 fatcat:g2bo6ybu65cvngzp5lllwo2isu

GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records [article]

Xi Yang, Nima Pour Nejatian, Hoo Chang Shin, Kaleb Smith, Christopher Parisien, Colin Compas, cheryl Martin, Mona Flores, Ying Zhang, Tanja Magoc, Christopher Harle, Gloria Lipori (+5 others)
2022 medRxiv   pre-print
In this study, we developed a large clinical transformer model - GatorTron - using >90 billion words of text and evaluated it on 5 clinical NLP tasks including clinical concept extraction, relation extraction  ...  GatorTron is now the largest transformer model in the clinical domain that scaled up from the previous 110 million to 8.9 billion parameters and achieved state-of-the-art performance on the 5 clinical  ...  It is unclear how transformer-based models developed using significantly more clinical narrative text and more parameters may improve medical AI systems in extracting and utilizing patient information.  ... 
doi:10.1101/2022.02.27.22271257 fatcat:mqbbbce4wre5bbgdbkqd7o44fq

GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records [article]

Xi Yang, Nima PourNejatian, Hoo Chang Shin, Kaleb E Smith, Christopher Parisien, Colin Compas, Cheryl Martin, Mona G Flores, Ying Zhang, Tanja Magoc, Christopher A Harle, Gloria Lipori (+5 others)
2022 arXiv   pre-print
models in the clinical and biomedical domain on 5 different clinical NLP tasks including clinical concept extraction, relation extraction, semantic textual similarity, natural language inference, and  ...  Objective: To develop a large pretrained clinical language model from scratch using transformer architecture; systematically examine how transformer models of different sizes could help 5 clinical natural  ...  It is unclear how transformer-based models developed using significantly more clinical narrative text and more parameters may improve medical AI systems in extracting and utilizing patient information.  ... 
arXiv:2203.03540v2 fatcat:xorxyl7xzzfpvolwwnkmxgvuky

Clinical-Longformer and Clinical-BigBird: Transformers for long clinical sequences [article]

Yikuan Li, Ramsey M. Wehbe, Faraz S. Ahmad, Hanyin Wang, Yuan Luo
2022 arXiv   pre-print
Transformers-based models, such as BERT, have dramatically improved the performance for various natural language processing tasks.  ...  Inspired by the success of these long sequence transformer models, we introduce two domain enriched language models, namely Clinical-Longformer and Clinical-BigBird, which are pre-trained from large-scale  ...  and document classification. 2 Related Work Clinical and Biomedical Transformers Transformer-based models, especially BERT [1] , can be enriched with clinical and biomedical knowledge through pre-training  ... 
arXiv:2201.11838v3 fatcat:pnqj4u24abgw7omjjhwdpr2xxu

AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models [article]

Katikapalli Subramanyam Kalyan, Ajit Rajasekharan, Sivanesan Sangeetha
2021 arXiv   pre-print
Transformer-based pretrained language models (PLMs) have started a new era in modern natural language processing (NLP).  ...  We introduce a taxonomy for transformer-based BPLMs and then discuss all the models. We discuss various challenges and present possible solutions.  ...  Entity Extraction is useful in many tasks like entity linking, relation extraction, knowledge graph construction, etc.  ... 
arXiv:2105.00827v2 fatcat:yzsr4tg7lrexzinrn5psw5r5q4

Clinical Trial Information Extraction with BERT [article]

Xiong Liu, Greg L. Hersch, Iya Khalil, Murthy Devarakonda
2021 arXiv   pre-print
Natural language processing (NLP) of clinical trial documents can be useful in new trial design.  ...  We trained named entity recognition (NER) models to extract eligibility criteria entities by fine-tuning a set of pre-trained BERT models.  ...  The relation extraction module is used to associate attribute entities with their base entities, which will be discussed in the extended version of the paper.  ... 
arXiv:2110.10027v1 fatcat:nzti3tfqp5gjfbvoxq6o2qm73y

A Study of Social and Behavioral Determinants of Health in Lung Cancer Patients Using Transformers-based Natural Language Processing Models [article]

Zehao Yu, Xi Yang, Chong Dang, Songzi Wu, Prakash Adekkanattu, Jyotishman Pathak, Thomas J. George, William R. Hogan, Yi Guo, Jiang Bian, Yonghui Wu
2021 arXiv   pre-print
In this study, we examined two state-of-the-art transformer-based NLP models, including BERT and RoBERTa, to extract SBDoH concepts from clinical narratives, applied the best performing model to extract  ...  analyses and machine learning-based models.  ...  Prevention) 1U18DP006512-01, the University of Florida (UF) SEED Program (DRPD-ROF2020, P0175580), and the Cancer Informatics and eHealth core jointly supported by the UF Health Cancer Center and the UF Clinical  ... 
arXiv:2108.04949v1 fatcat:pt4yhw5gf5eovotwt7y5a2ay3e

Using Clinical Drug Representations for Improving Mortality and Length of Stay Predictions [article]

Batuhan Bardak, Mehmet Tan
2021 arXiv   pre-print
Time-varying vital signs, laboratory measurements, and related time-series signals are commonly used to predict clinical outcomes.  ...  models.  ...  We propose a pipeline to extract and transform drug information into useful vector representations to be used in deep learning models.  ... 
arXiv:2110.08918v1 fatcat:kmhbgsnt25bodjwp6xuigsct7a

Clinical data interoperability based on archetype transformation

Catalina Martínez Costa, Marcos Menárguez-Tortosa, Jesualdo Tomás Fernández-Breis
2011 Journal of Biomedical Informatics  
In this work, we have focused on the semantic interoperability of electronic healthcare records standards based on the dual model architecture and we have developed a solution that has been applied to  ...  The semantic interoperability between health information systems is a major challenge to improve the quality of clinical practice and patient safety.  ...  Special thanks to Daniel Karlsson, Marcelo Rodrigues dos Santos and Diego Boscá for providing us with data extracts for the evaluation of our method.  ... 
doi:10.1016/j.jbi.2011.05.006 pmid:21645637 fatcat:5k3vgrv2kjfvnfknykv6dxrzqa

Integrating shortest dependency path and sentence sequence into a deep learning framework for relation extraction in clinical text

Zhiheng Li, Zhihao Yang, Chen Shen, Jun Xu, Yaoyun Zhang, Hua Xu
2019 BMC Medical Informatics and Decision Making  
Researchers have applied deep learning-based approaches to clinical relation extraction; but most of them consider sentence sequence only, without modeling syntactic structures.  ...  Extracting relations between important clinical entities is critical but very challenging for natural language processing (NLP) in the medical domain.  ...  for clinical relation extraction.  ... 
doi:10.1186/s12911-019-0736-9 pmid:30700301 pmcid:PMC6354333 fatcat:pz6tz64annbmdny5u47pjjb32a

An OMOP CDM-Based Relational Database of Clinical Research Eligibility Criteria

Yuqi Si, Chunhua Weng
2017 Studies in Health Technology and Informatics  
In this paper, we described a practical method for transforming free-text clinical research eligibility criteria of Alzheimer's clinical trials into a structured relational database compliant with standards  ...  for medical terminologies and clinical data models.  ...  Acknowledgments This research was supported by R01 LM009886 from The National Library of Medicine (Bridging the semantic gap between research eligibility criteria and clinical data; PI:Weng).  ... 
pmid:29295240 pmcid:PMC5893219 fatcat:osh3dcpvarhbhb7waaejgtih3e

Adverse drug events and medication relation extraction in electronic health records with ensemble deep learning methods

2019 JAMIA Journal of the American Medical Informatics Association  
We additionally developed separate models for intra- and inter-sentence relation extraction and combined them using an ensemble method.  ...  We incorporated state-of-the-art named-entity recognition (NER) models based on bidirectional long short-term memory (BiLSTM) networks and conditional random fields (CRF) for end-to-end extraction.  ...  To the best of our knowledge, this is the first time Transformer is used for mention-level RE in clinical records. Our team ranked third in both relation and end-to-end extraction tasks.  ... 
doi:10.1093/jamia/ocz101 pmid:31390003 pmcid:PMC6913215 fatcat:l2ciicjt6zaqniafn7ibkfef3e

Towards Generic MDE Support for Extracting Purpose-Specific Healthcare Models from Annotated, Unstructured Texts [chapter]

Pieter Van Gorp, Irene Vanderfeesten, Willem Dalinghaus, Josh Mengerink, Bram van der Sanden, Pieter Kubben
2013 Lecture Notes in Computer Science  
Once healthcare-specific models have been captured formally (i.e., in a metamodel-based language), the application of model transformation, analysis and code generation techniques is rather straightforward  ...  research challenge to build a robust and generic (i.e., metamodel-independent) tool for this important type of model extraction support.  ...  Conclusions This short paper focuses on a specific MDE contribution: the development of a CDS based on the extraction of models from unstructured clinical guideline texts.  ... 
doi:10.1007/978-3-642-39088-3_14 fatcat:65ze5xkiqnhhlklo3nah5d4wbe

Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review [article]

Irene Li, Jessica Pan, Jeremy Goldwasser, Neha Verma, Wai Pan Wong, Muhammed Yavuz Nuzumlalı, Benjamin Rosand, Yixin Li, Matthew Zhang, David Chang, R. Andrew Taylor, Harlan M. Krumholz (+1 others)
2021 arXiv   pre-print
Well over half of the information stored within EHRs is in the form of unstructured text (e.g. provider notes, operation reports) and remains largely untapped for secondary use.  ...  Recently, however, newer neural network and deep learning approaches to Natural Language Processing (NLP) have made considerable advances, outperforming traditional statistical and rule-based systems on  ...  [41] employed a similar BiLSTM-CRF model for intra-sentence relation extraction, but used a Transformer-based model to capture longer dependencies in modeling inter-sentence relations.  ... 
arXiv:2107.02975v1 fatcat:nayhw7gadfdzrovycdkvzy75pi

End-to-End Approach for Structuring Radiology Reports

Kento Sugimoto, Toshihiro Takeda, Shoya Wada, Asuka Yamahata, Shozo Konishi, Shiro Manabe, Yasushi Matsumura
2020 Studies in Health Technology and Informatics  
First, we built a neural network model to extract clinical information from the reports. We experimented on a dataset of chest X-ray reports.  ...  Radiology reports include various types of clinical information that are used for patient care.  ...  Specifically, we take three steps: extraction, structuring and normalization. In the first step, we build a recurrent neural network-based model for entity recognition.  ... 
doi:10.3233/shti200151 pmid:32570375 fatcat:oxarupwuevb3rj3ndjqkfpy3pe
« Previous Showing results 1 — 15 out of 334,371 results