6,497 Hits in 3.9 sec

Multimodal Self-Supervised Learning for Medical Image Analysis [article]

Aiham Taleb, Christoph Lippert, Tassilo Klein, Moin Nabi
2020 arXiv   pre-print
Our results also highlight the benefits of exploiting synthetic images for self-supervised pretraining.  ...  In addition, we also propose to utilize cross-modal generation techniques for multimodal data augmentation used for training self-supervised tasks.  ...  Self-supervision in the medical context.  ... 
arXiv:1912.05396v2 fatcat:klwchysg4ney3phlmlawyrzxvi

Learning the retinal anatomy from scarce annotated data using self-supervised multimodal reconstruction

Álvaro S. Hervella, José Rouco, Jorge Novo, Marcos Ortega
2020 Applied Soft Computing  
For that purpose, a neural network is pre-trained using the self-supervised multimodal reconstruction of fluorescein angiography from retinography.  ...  Deep learning is becoming the reference paradigm for approaching many computer vision problems.  ...  A rich source of information that has still not been exploited for self-supervised transfer learning is the unlabelled multimodal data in medical imaging.  ... 
doi:10.1016/j.asoc.2020.106210 fatcat:7zr5xxza6rgald67fkcnve557y

Self-Supervised Multimodal Domino: in Search of Biomarkers for Alzheimer's Disease [article]

Alex Fedorov, Tristan Sylvain, Eloy Geenjaar, Margaux Luck, Lei Wu, Thomas P. DeRamus, Alex Kirilin, Dmitry Bleklov, Vince D. Calhoun, Sergey M. Plis
2021 arXiv   pre-print
In this paper, we unify recent work on multimodal self-supervised learning under a single framework.  ...  This observation motivated the design of powerful multimodal self-supervised representation-learning algorithms.  ...  ., different MRI modalities [1] , [2] in medical imaging, LiDAR, and video for self-driving cars [3] , and confounder-influenced data [4] .  ... 
arXiv:2012.13623v4 fatcat:ndgnj5pakvgkfpoq55bvfz2zli

Self-supervised multimodal reconstruction of retinal images over paired datasets

Álvaro S. Hervella, José Rouco, Jorge Novo, Marcos Ortega
2020 Expert systems with applications  
This work proposes a novel self-supervised multimodal reconstruction task that takes advantage of this unlabeled multimodal data for learning about the domain without human supervision.  ...  These results indicate that the proposed self-supervised task provides relevant cues for image analysis tasks in the same domain.  ...  Proposed work The proposed self-supervised multimodal reconstruction paradigm naturally fits medical image applications, given the extensive use of multimodal visual data in many clinical specialties.  ... 
doi:10.1016/j.eswa.2020.113674 fatcat:2x7q6ag4xjf73oukbinjwhvvga

Tasting the cake: evaluating self-supervised generalization on out-of-distribution multimodal MRI data [article]

Alex Fedorov, Eloy Geenjaar, Lei Wu, Thomas P. DeRamus, Vince D. Calhoun, Sergey M. Plis
2022 arXiv   pre-print
Self-supervised learning has enabled significant improvements on natural image benchmarks. However, there is less work in the medical imaging domain in this area.  ...  Our results highlight the need for out-of-distribution generalization standards and benchmarks to adopt the self-supervised methods in the medical imaging community.  ...  CONCLUSIONSSelf-supervised medical imaging models are only now beginning to be developed, and we hope that our analysis will facilitate robust and fair self-supervised models.  ... 
arXiv:2103.15914v3 fatcat:rre34oatsjhcjmpk2xw7ftjcua

Unsupervised MRI Super-Resolution Using Deep External Learning and Guided Residual Dense Network with Multimodal Image Priors [article]

Yutaro Iwamoto, Kyohei Takeda, Yinhao Li, Akihiko Shiino, Yen-Wei Chen
2020 arXiv   pre-print
These techniques have also been applied to medical image super-resolution (SR). Compared with natural images, medical images have several unique characteristics.  ...  Deep learning techniques have led to state-of-the-art single image super-resolution (SISR) with natural images.  ...  The authors would like to thank Enago ( for the English language review.  ... 
arXiv:2008.11921v2 fatcat:uls762ztunclbln3xa5xeelv2u

Learning Retinal Patterns from Multimodal Images

Álvaro S. Hervella, José Rouco, Jorge Novo, Marcos Ortega
2018 Proceedings (MDPI)  
The self-supervised training of a reconstruction task between paired multimodal images can be used to learn about the image contents without using any label.  ...  In this work, we propose the use of complementary medical image modalities as an alternative to reduce the required annotated data.  ...  We train a neural network to predict the angiography from a retinography of the same patient and demonstrate that the network learns about relevant structures of the eye with this self-supervised training  ... 
doi:10.3390/proceedings2181195 fatcat:sv2ba5vq5vavplnl6s4ihwp2ou

MMBERT: Multimodal BERT Pretraining for Improved Medical VQA [article]

Yash Khare, Viraj Bagal, Minesh Mathew, Adithi Devi, U Deva Priyakumar, CV Jawahar
2021 arXiv   pre-print
To overcome these limitations, we propose a solution inspired by self-supervised pretraining of Transformer-style architectures for NLP, Vision and Language tasks.  ...  Our method involves learning richer medical image and text semantic representations using Masked Language Modeling (MLM) with image features as the pretext task on a large medical image+caption dataset  ...  Since the annotations on medical images require the help of an expert, it is difficult to crowdsource and annotation cost is high. This motivates the usage of self-supervised pretraining methods.  ... 
arXiv:2104.01394v1 fatcat:nfgvcaftzjgslec2fuujidsm5e

Front Matter: Volume 12032

Ivana Išgum, Olivier Colliot
2022 Medical Imaging 2022: Image Processing  
2D KV/MV image [12032-124] 3K Size- [12032-105] 31 Bridging the domain gap for medical image segmentation with multimodal MIND features [12032-106] 32 Benefits of auxiliary information in deep learning-based  ...  for deep learning segmentation in medical imaging [12032-25] 10 Unsupervised domain adaptation for segmentation with black-box source model [12032-26] 11 Do I know this?  ... 
doi:10.1117/12.2638192 fatcat:ikfgnjefaba2tpiamxoftyi6sa

Deep Learning in Medical Image Registration: A Survey [article]

Grant Haskins, Uwe Kruger, Pingkun Yan
2019 arXiv   pre-print
Since the beginning of the recent deep learning renaissance, the medical imaging research community has developed deep learning based approaches and achieved the state-of-the-art in many applications,  ...  The rapid adoption of deep learning for image registration applications over the past few years necessitates a comprehensive summary and outlook, which is the main scope of this survey.  ...  Weakly-supervised convolutional neural networks for multimodal image registration. Medical image analysis, 49:1-13. 53. Ikeda, K., Ino, F., and Hagihara, K. (2014).  ... 
arXiv:1903.02026v1 fatcat:6ulnzrbj6rb55eydtkgygg5r6u

Medical Image Registration Using Deep Neural Networks: A Comprehensive Review [article]

Hamid Reza Boveiri, Raouf Khayami, Reza Javidan, Ali Reza MehdiZadeh
2020 arXiv   pre-print
In this paper, a comprehensive review on the state-of-the-art literature known as medical image registration using deep neural networks is presented.  ...  to challenge with many medical applications, where the registration is not an exception.  ...  Image Processing, and Ultrasound Systems for Assisted Diagnosis and Navigation 1 Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support Image Analysis for Moving  ... 
arXiv:2002.03401v1 fatcat:u4utrifr2rg3bf6x6fgohyfmpy

2020 Index IEEE Transactions on Artificial Intelligence Vol. 1

2020 IEEE Transactions on Artificial Intelligence  
., +, TAI Oct. 2020 181-191 Image motion analysis Self-Supervised Pose Adaptation for Cross-Domain Image Animation.  ...  ., +, TAI Aug. 2020 62-73 Self-Supervised Pose Adaptation for Cross-Domain Image Animation.  ... 
doi:10.1109/tai.2021.3089904 fatcat:53o6433ljne3lblvr5fuy66lou

Brain MR Multimodal Medical Image Registration Based on Image Segmentation and Symmetric Self-similarity

2020 KSII Transactions on Internet and Information Systems  
More particularly, we propose two novel symmetric self-similarity constraint operators to constrain the segmented medical images and convert each modal medical image into a unified modal for multimodal  ...  In this paper, a multimodal medical image registration method based on image segmentation and symmetric self-similarity is proposed.  ...  The unsupervised piece of self-supervised learning based on similarity measures is currently a research hotspot.  ... 
doi:10.3837/tiis.2020.03.014 fatcat:bvaxau4x6fbcxki2avgdncexsi

A Review of Self-supervised Learning Methods in the Field of Medical Image Analysis

Jiashu Xu, National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute", Kyiv, 03056, Ukraine
2021 International Journal of Image Graphics and Signal Processing  
So, more and more researchers are trying to utilize SSL methods for medical image analysis, to meet the challenge of assembling large medical datasets.  ...  review the application of self-supervised learning in the medical field.  ...  Acknowledgment This research has been partially supported by China Scholarship Council (CSC), and Special thanks should go to my supervisor professor Sergii Stirenko, for his instructive advice and useful  ... 
doi:10.5815/ijigsp.2021.04.03 fatcat:ff7ybaplqncthgswf3zy7cbeza

Enhancing Retinal Blood Vessel Segmentation through Self-Supervised Pre-Training

José Morano, Álvaro S. Hervella, Noelia Barreira, Jorge Novo, José Rouco
2020 Proceedings (MDPI)  
In this work, we present a novel application of self-supervised multimodal pre-training to enhance the retinal vasculature segmentation.  ...  Table 1 . 1 Best AUC-ROC and AUC-PR values of the different networks trained from scratch (FS) and using self-supervised multimodal pretraining (SSMP) for the STARE dataset.  ...  This motivates the proposal of self-supervised multimodal pre-training (SSMP) to learn the relevant patterns from unlabeled data and reduce the required amount of annotated data [1] [2] [3] .  ... 
doi:10.3390/proceedings2020054044 fatcat:crkjipbjcfe33cmg6y3jcqzrmy
« Previous Showing results 1 — 15 out of 6,497 results