13 Hits in 10.7 sec

VQA-Med: Overview of the Medical Visual Question Answering Task at ImageCLEF 2019

Asma Ben Abacha, Sadid A. Hasan, Vivek V. Datla, Joey Liu, Dina Demner-Fushman, Henning Müller
2019 Conference and Labs of the Evaluation Forum  
This paper presents an overview of the Medical Visual Question Answering task (VQA-Med) at ImageCLEF 2019.  ...  Participating systems were tasked with answering medical questions based on the visual content of radiology images.  ...  Acknowledgments This work was supported by the intramural research program at the U.S. National Library of Medicine, National Institutes of Health. We thank Dr. James G.  ... 
dblp:conf/clef/AbachaHDLDM19 fatcat:gpu5qdvcvzgdnpfwp2is5i7gwa

SSN MLRG at VQA-MED 2021: An Approach for VQA to Solve Abnormality Related Queries using Improved Datasets

Noor Mohamed Sheerin Sitara, Kavitha Srinivasan
2021 Conference and Labs of the Evaluation Forum  
The Visual Question Answering (VQA) in the medical domain attains tremendous advancement in last few years.  ...  These four datasets are build by collecting the samples from previous ImageCLEF VQA -MED tasks.  ...  The overview of ImageCLEF VQA -MED tasks (2018, 2019 and 2020) are summarized and given in Table 1 . From the results, the observations are: (i).  ... 
dblp:conf/clef/SitaraS21 fatcat:7j224ar465fj5nyzr3btjdkiiq

Overview of the VQA-Med Task at ImageCLEF 2021: Visual Question Answering and Generation in the Medical Domain

Asma Ben Abacha, Mourad Sarrouti, Dina Demner-Fushman, Sadid A. Hasan, Henning Müller
2021 Conference and Labs of the Evaluation Forum  
This paper presents an overview of the fourth edition of the Medical Visual Question Answering (VQA-Med) task at ImageCLEF 2021.  ...  VQA-Med 2021 includes a task on Visual Question Answering (VQA), where participants are tasked with answering questions from the visual content of radiology images, and a second task on Visual Question  ...  Acknowledgments This work was partially supported by the intramural research program at the U.S. National Library of Medicine, National Institutes of Health.  ... 
dblp:conf/clef/AbachaSDHM21 fatcat:76fzt5dmcfentdqkhturjsci4y

SYSU-HCP at VQA-Med 2021: A Data-centric Model with Efficient Training Methodology for Medical Visual Question Answering

Haifan Gong, Ricong Huang, Guanqi Chen, Guanbin Li
2021 Conference and Labs of the Evaluation Forum  
This paper describes our contribution to the Visual Question Answering Task in the Medical Domain at ImageCLEF 2021.  ...  Our code and model are available at  ...  To facilitate the lack of the benchmark in the medical VQA, ImageCLEF organizes the 4th edition of the Medical Domain Visual Question Answering Competition named VQA-Med 2021.  ... 
dblp:conf/clef/GongHCL21 fatcat:ukuxhucuy5d2ncw6xsvsicom4i

Ensemble of Streamlined Bilinear Visual Question Answering Models for the ImageCLEF 2019 Challenge in the Medical Domain

Minh H. Vu, Raphael Sznitman, Tufve Nyholm, Tommy Löfstedt
2019 Conference and Labs of the Evaluation Forum  
for the Medical Domain Visual Question Answering challenge hosted by ImageCLEF 2019.  ...  The proposed method was ranked 3rd in the Medical Domain Visual Question Answering challenge of Im-ageCLEF 2019.  ...  Last, we presented an ensemble method that boosted the performance. Fig. 1 : 1 Fig. 1: Examples of questions and images and their corresponding answers in the ImageCLEF-VQA-Med 2019 challenge.  ... 
dblp:conf/clef/VuSNL19 fatcat:dm6zj7k2g5hqtd5zcpazxou6te

CGMVQA: A new Classification and Generative Model for Medical Visual Question Answering

Fuji Ren, Yangyang Zhou
2020 IEEE Access  
This model establishes new state-of-the-art results: 0.640 of classification accuracy, 0.659 of word matching and 0.678 of semantic similarity in ImageCLEF 2019 VQA-Med data set.  ...  It suggests that the CGMVQA is effective in medical visual question answering and can better assist doctors in clinical analysis and diagnosis.  ...  We use the ImageCLEF 2019 VQA-Med data set here.  ... 
doi:10.1109/access.2020.2980024 fatcat:liiwta2vfrazbnfiiymzdf4d3i

ImageCLEF 2020: Multimedia Retrieval in Lifelogging, Medical, Nature, and Internet Applications [chapter]

Bogdan Ionescu, Henning Müller, Renaud Péteri, Duc-Tien Dang-Nguyen, Liting Zhou, Luca Piras, Michael Riegler, Pål Halvorsen, Minh-Triet Tran, Mathias Lux, Cathal Gurrin, Jon Chamberlain (+20 others)
2020 Lecture Notes in Computer Science  
This paper presents an overview of the 2020 ImageCLEF lab that will be organized as part of the Conference and Labs of the Evaluation Forum-CLEF Labs 2020 in Thessaloniki, Greece.  ...  ImageCLEF is an ongoing evaluation initiative (run since 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access  ...  The medical Visual Question Answering (VQA-Med) task poses a challenging problem that involves both natural language processing and computer vision.  ... 
doi:10.1007/978-3-030-45442-5_69 fatcat:fpczf3xzrfefbc5rwykvqf4mfm

Medical Visual Question Answering: A Survey [article]

Zhihong Lin, Donghao Zhang, Qingyi Tac, Danli Shi, Gholamreza Haffari, Qi Wu, Mingguang He, Zongyuan Ge
2022 arXiv   pre-print
Medical Visual Question Answering~(VQA) is a combination of medical artificial intelligence and popular VQA challenges.  ...  Given a medical image and a clinically relevant question in natural language, the medical VQA system is expected to predict a plausible and convincing answer.  ...  VQA-Med-2019 VQA-Med-2019 [14] is the second edition of the VQA-Med and was published during the ImageCLEF 2019 challenge.  ... 
arXiv:2111.10056v2 fatcat:4dihtqmptbgj5lozrv3lfxqv7q

Hierarchical Deep Multi-modal Network for Medical Visual Question Answering [article]

Deepak Gupta, Swati Suman, Asif Ekbal
2020 arXiv   pre-print
Visual Question Answering in Medical domain (VQA-Med) plays an important role in providing medical assistance to the end-users.  ...  proper answers to the queries related to medical images; and thirdly, we study the impact of QS in Medical-VQA by comparing the performance of the proposed model with QS and a model without QS.  ...  Recently, the ImageCLEF introduced the challenge of Medical Domain Visual Question Answering, VQA-Med 2018 2 (Ionescu et al., 2018).  ... 
arXiv:2009.12770v1 fatcat:d2dmtduat5b3bm4ujgyryh474y


Yefan Huang, Xiaoli Wang, Feiyan Liu, Guofeng Huang
2022 Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval  
Medical visual question answering (Med-VQA) is a challenging problem that aims to take a medical image and a clinical question about the image as input and output a correct answer in natural language.  ...  To evaluate the quality of OVQA, we conduct comprehensive experiments on state-of-the-art methods for the Med-VQA task to our dataset.  ...  We thank the Affiliated Southeast Hospital of Xiamen University for providing us with electronic medical records. We appreciate Ms.  ... 
doi:10.1145/3477495.3531724 fatcat:smrinohe6fh4bgeoz4zpmu6j44

A Comparison of Pre-trained Vision-and-Language Models for Multimodal Representation Learning across Medical Images and Reports [article]

Yikuan Li, Hanyin Wang, Yuan Luo
2020 arXiv   pre-print
Joint image-text embedding extracted from medical images and associated contextual reports is the bedrock for most biomedical vision-and-language (V+L) tasks, including medical visual question answering  ...  We also visualize attention maps to illustrate the attention mechanism of V+L models.  ...  [20] won the first place in ImageCLEF 2019 VQA-Med [21] competition using the "VGG16+BERT+MFB" model.  ... 
arXiv:2009.01523v1 fatcat:ktrgckfombbdxmafis2dftdnr4

ViLMedic: a framework for research at the intersection of vision and language in medical AI

Jean-benoit Delbrouck, Khaled Saab, Maya Varma, Sabri Eyuboglu, Pierre Chambon, Jared Dunnmon, Juan Zambrano, Akshay Chaudhari, Curtis Langlotz
2022 Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations   unpublished
As of 2022, the library contains a dozen reference implementations replicating the state-of-the-art results for problems that range from medical visual question answering and radiology report generation  ...  The library is available at https://github. com/jbdel/vilmedic.  ...  Medical Visual Question Answering VQA in the medical domain consists of building systems that answer open-ended questions about medical images ranging from x-rays, MRI to CT scans.  ... 
doi:10.18653/v1/2022.acl-demo.3 fatcat:lmmxnicjxfg5vgucoby63bjoja

Towards Visual Dialog for Radiology

Olga Kovaleva, Chaitanya Shivade, Satyananda Kashyap, Karina Kanjaria, Joy Wu, Deddeh Ballah, Adam Coy, Alexandros Karargyris, Yufan Guo, David Beymer Beymer, Anna Rumshisky, Vandana Mukherjee Mukherjee
2020 Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing   unpublished
We show that incorporating medical history of the patient leads to better performance in answering questions as opposed to conventional visual question answering model which looks only at the image.  ...  To address this limitation, we introduce a realistic and information-rich task of Visual Dialog in radiology, specific to chest X-ray images.  ...  SAN has been successfully adapted for medical VQA tasks such as VQA-RAD (Lau et al., 2018) and VQA-Med task of the ImageCLEF 2018 challenge 2019 Figure 3 : 20193 For example, the pairs {'Other pleural  ... 
doi:10.18653/v1/2020.bionlp-1.6 fatcat:vcugncpkobb2nayzjgrslhsh44