SSN MLRG at VQA-MED 2021: An Approach for VQA to Solve Abnormality Related Queries using Improved Datasets

Noor Mohamed Sheerin Sitara, Kavitha Srinivasan
2021 Conference and Labs of the Evaluation Forum  
The Visual Question Answering (VQA) in the medical domain attains tremendous advancement in last few years. To improvise the VQA research, ImageCLEF forum is organizing the fourth edition of VQA task in medical domain. This year, the abnormality related VQA queries are to be answered for the given set of radiology images. In the proposed system, VGGNet based on transfer learning approach and LSTM is used to extract image and text features respectively. The extracted three dimensional
more » ... image, text) feature vectors are concatenated into sequence of vectors by LSTM for predicting the answer. The purpose of selecting VGGNet and LSTM are: VGGNet, outperforms complex recognition tasks and also addresses vanishing gradient and exploding gradient problem and LSTM, solves complex sequence learning problems and overcomes long term dependency problems. In addition, the hyper parameters are chosen appropriately and four improved datasets are used to analyze the performance of the proposed model. These four datasets are build by collecting the samples from previous ImageCLEF VQA -MED tasks. The proposed model resulted in an accuracy of 0.196 and a BLEU score of 0.227 for one of the dataset, which is ranked tenth among all participating groups in ImageCLEF 2021 VQA-MED task.
dblp:conf/clef/SitaraS21 fatcat:7j224ar465fj5nyzr3btjdkiiq