2,307 Hits in 3.1 sec

Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines

Shih-Cheng Huang, Anuj Pareek, Saeed Seyyedi, Imon Banerjee, Matthew P. Lungren
2020 npj Digital Medicine  
We conducted a systematic search on PubMed and Scopus for original research articles leveraging deep learning for fusion of multimodality data.  ...  fusion in medical imaging.  ...  ACKNOWLEDGEMENTS The authors wish to thank John Alexander Borghi from Stanford Lane Medical Library for his help with creating the systematic search.  ... 
doi:10.1038/s41746-020-00341-z pmid:33083571 pmcid:PMC7567861 fatcat:vo7os2ial5eqzju3ere3x2mgwi

Multimodal Deep Learning for Cervical Dysplasia Diagnosis [chapter]

Tao Xu, Han Zhang, Xiaolei Huang, Shaoting Zhang, Dimitris N. Metaxas
2016 Lecture Notes in Computer Science  
Our multimodal framework is an end-to-end deep network which can learn better complementary features from the image and non-image modalities.  ...  We first employ the convolutional neural network (CNN) to convert the low-level image data into a feature vector fusible with other non-image modalities.  ...  The fusion between image and non-image modalities only happened when merging the decision scores from each modality. We call this type of fusion as Late Fusion.  ... 
doi:10.1007/978-3-319-46723-8_14 fatcat:k42sdzwux5g5jg3fka3s4bpgeu

Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology images [article]

Alexander Partin
2022 arXiv   pre-print
The MM-Net learns to predict response using drug descriptors, gene expressions (GE), and histology whole-slide images (WSIs) where the multi-modality refers to the tumor features.  ...  Our results show that data augmentation and integration of histology images with GE can improve prediction performance of drug response in PDXs.  ...  Therefore, multimodal learning exhibits tradeoff between enriching the feature space via multimodal fusion and overfitting.  ... 
arXiv:2204.11678v1 fatcat:bcplavqsujfqlpz3emkiww6oj4

Two-stage Spatial Mapping for Multimodal Data Fusion in Mobile Crowd Sensing

Jiancun Zhou, Tao Xu, Sheng Ren, Kehua Guo
2020 IEEE Access  
In order to process multimodal data better, we propose a fusion and classification method for multimodal data.  ...  Then, through bilinear pooling, the representations of different modality are fused, and the fused vectors are used in the classification task.  ...  to fuse different histological images [28] .  ... 
doi:10.1109/access.2020.2995268 fatcat:6fzbrjikbjbubkyh42rchuh444

Spectral embedding-based registration (SERg) for multimodal fusion of prostate histology and MRI

Eileen Hwuang, Mirabela Rusu, Sudha Karthigeyan, Shannon C. Agner, Rachel Sparks, Natalie Shih, John E. Tomaszewski, Mark Rosen, Michael Feldman, Anant Madabhushi, Sebastien Ourselin, Martin A. Styner
2014 Medical Imaging 2014: Image Processing  
images to facilitate multimodal image registration.  ...  In this work, SERg is implemented using Demons to allow the algorithm to more effectively register multimodal images.  ...  The approach was successfully applied to synthetic brain images and prostate histology-MRI fusion.  ... 
doi:10.1117/12.2044317 dblp:conf/miip/HwuangRKASSTRFM14 fatcat:4saaiovoyfgrddhfuq7vkcovoa

Fast computation of mutual information in the frequency domain with applications to global multimodal image alignment [article]

Johan Öfverstedt, Joakim Lindblad, Nataša Sladoje
2021 arXiv   pre-print
Multimodal image alignment is the process of finding spatial correspondences between images formed by different imaging techniques or under different conditions, to facilitate heterogeneous data fusion  ...  We evaluate the efficacy of the proposed method on three distinct benchmark datasets, of aerial images, cytological images, and histological images, and we observe excellent success-rates (in recovering  ...  Table 1 : 1 Run-time in seconds for global rigid image alignment for a number of configurations using images taken from the cytological and histological dataset.  ... 
arXiv:2106.14699v1 fatcat:g7olg3jihjcf7buogitmdp3ura

Combined MR, fluorescence and histology imaging strategy in a human breast tumor xenograft model

Lu Jiang, Tiffany R. Greenwood, Erika R. Amstalden van Hove, Kamila Chughtai, Venu Raman, Paul T. Winnard, Ron M. A. Heeren, Dmitri Artemov, Kristine Glunde
2012 NMR in Biomedicine  
We have developed a multimodal image reconstruction and fusion framework that accurately combines in vivo magnetic resonance imaging (MRI) and magnetic resonance spectroscopic imaging (MRSI), ex vivo brightfield  ...  Ex vivo brightfield microscopic imaging was used as an intermediate modality to facilitate the ultimate link between ex vivo histology and in vivo MRI/MRSI.  ...  Bhujwalla for useful discussions. This work was supported by the National Institutes of Health (NIH) grant R01 CA134695.  ... 
doi:10.1002/nbm.2846 pmid:22945331 pmcid:PMC4162316 fatcat:u33kktkzq5euhhmtfqjsllcugu

Registration and Fusion of the Autofluorescent and Infrared Retinal Images

Radim Kolar, Libor Kubecka, Jiri Jan
2008 International Journal of Biomedical Imaging  
This article deals with registration and fusion of multimodal opththalmologic images obtained by means of a laser scanning device (Heidelberg retina angiograph).  ...  The registration framework has been designed and tested for combination of autofluorescent and infrared images.  ...  The authors highly acknowledge availability of a test set of images provided by the Department of Ophthalmology, Friedrich-Alexander University of Erlangen-Nurnberg (Germany).  ... 
doi:10.1155/2008/513478 pmid:18949055 pmcid:PMC2570800 fatcat:unfl24x7zzhlnmrafyuo4q3lji

Fusing Heterogeneous Data for Alzheimer's Disease Classification

Parvathy Sudhir Pillai, Tze-Yun Leong, Alzheimer's Disease Neuroimaging Initiative
2015 Studies in Health Technology and Informatics  
Our results indicate that multimodal data fusion improves classification accuracy.  ...  In multi-view learning, multimodal representations of a real world object or situation are integrated to learn its overall picture.  ...  Data used in the preparation of this article were obtained from the ADNI database (  ... 
pmid:26262148 fatcat:hwhycydu7rcz5bqenbyewbavyy

Integrated diagnostics: a conceptual framework with examples

Anant Madabhushi, Scott Doyle, George Lee, Ajay Basavanhally, James Monaco, Steve Masters, John Tomaszewski, Michael Feldman
2010 Clinical Chemistry and Laboratory Medicine  
histological image and proteomic signatures for prostate cancer outcome prediction.  ...  In addition, we discuss some exciting recent developments in the application of these methods for multi-modal data fusion and classification; specifically the building of meta-classifiers by fusion of  ...  Generalized fusion framework H&E stained prostate whole mount histology and corresponding proteomic spectra from mass spectrometry can be used to predict prostate cancer recurrence via a generalized fusion  ... 
doi:10.1515/cclm.2010.193 pmid:20491597 fatcat:q2dseyj7ivgu7pndbtspni5354

Non-small-cell lung cancer classification via RNA-Seq and histology imaging probability fusion

Francisco Carrillo-Perez, Juan Carlos Morales, Daniel Castillo-Secilla, Yésica Molina-Castro, Alberto Guillén, Ignacio Rojas, Luis Javier Herrera
2021 BMC Bioinformatics  
In this work, we present a late fusion classification model using histology and RNA-Seq data for adenocarcinoma, squamous-cell carcinoma and healthy lung tissue.  ...  In recent years, there has been an increasing gathering of biological data for decision support systems in the diagnosis (e.g. histology imaging, next-generation sequencing technologies data, clinical  ...  Lung cancer histology imaging classification In recent years, the use of deep learning models for histology imaging classification has been taken into consideration based on the outstanding results that  ... 
doi:10.1186/s12859-021-04376-1 pmid:34551733 fatcat:keqooovgm5bmllyy3s5uouap4a

Artificial Intelligence-based methods in head and neck cancer diagnosis: an overview

Hanya Mahmood, Muhammad Shaban, Nasir Rajpoot, Syed A. Khurram
2021 British Journal of Cancer  
Methods Electronic database searches using MEDLINE via OVID, EMBASE and Google Scholar were conducted to retrieve articles using AI/ML for diagnostic evaluation of HNC (2009–2020).  ...  Imaging modalities included histological (n = 9), radiological (n = 8), hyperspectral (n = 6), endoscopic/clinical (n = 5), infrared thermal (n = 1) and optical (n = 1).  ...  ACKNOWLEDGEMENTS We thank the Health Sciences Library at The University of Sheffield for assistance with the electronic database search. AUTHOR CONTRIBUTIONS  ... 
doi:10.1038/s41416-021-01386-x pmid:33875821 pmcid:PMC8184820 fatcat:4jmkxrtosjbxrbe4glt5bz5psy

Integrating structural and functional imaging for computer assisted detection of prostate cancer on multi-protocol in vivo 3 Tesla MRI

Satish Viswanath, B. Nicolas Bloch, Mark Rosen, Jonathan Chappelow, Robert Toth, Neil Rofsky, Robert Lenkinski, Elizabeth Genega, Arjun Kalyanpur, Anant Madabhushi, Nico Karssemeijer, Maryellen L. Giger
2009 Medical Imaging 2009: Computer-Aided Diagnosis  
Our scheme is fully-automated and comprises (a) prostate segmentation, (b) multimodal image registration, and (c) data representation and multi-classifier modules for information fusion.  ...  cross validation was performed using a set of 18 images derived from 6 patient datasets on a per-pixel basis.  ...  Our recently developed multimodal image registration scheme, COLLINARUS, is used to register whole-mount histological sections and the multi-protocol MR data, as well as align T2 and DCE protocols prior  ... 
doi:10.1117/12.811899 pmid:25301989 pmcid:PMC4188347 fatcat:4ijiu4oki5ffzdgzc2gx6t3t5m

The Template of the Primary Lymphatic Landing Sites of the Prostate Should Be Revisited: Results of a Multimodality Mapping Study

Agostino Mattei, Frank G. Fuechsel, Nivedita Bhatta Dhar, Sebastian H. Warncke, George N. Thalmann, Thomas Krause, Urs E. Studer
2008 European Urology  
Furthermore, analysis of SPECT/CT/MRI fusion imaging requires the expertise of a nuclear medicine specialist dedicated to a thorough search for all nodes.  ...  ) fusion imaging confirmed by surgery.  ... 
doi:10.1016/j.eururo.2007.07.035 pmid:17709171 fatcat:463uug2w4jahheaornbxdy66jm

Supervised Regularized Canonical Correlation Analysis: integrating histologic and proteomic measurements for predicting biochemical recurrence following prostate surgery

Abhishek Golugula, George Lee, Stephen R Master, Michael D Feldman, John E Tomaszewski, David W Speicher, Anant Madabhushi
2011 BMC Bioinformatics  
These results suggest that SRCCA is a computationally efficient and a highly accurate scheme for representing multimodal (histologic and proteomic) data in a metaspace and that it could be used to construct  ...  We leverage this SRCCA framework towards the fusion of proteomic and histologic image signatures for identifying prostate cancer patients at the risk of 5 year biochemical recurrence following radical  ...  R01CA136535, R01CA140772, and R03CA143991), Department of Defense (W81XWH-08-1-0145), The Cancer Institute of New Jersey and the Society for Imaging Informatics in Medicine.  ... 
doi:10.1186/1471-2105-12-483 pmid:22182303 pmcid:PMC3267835 fatcat:3e5om4kcefgh5ciibvqxhxehse
« Previous Showing results 1 — 15 out of 2,307 results