How to Pre-Train Your Model? Comparison of Different Pre-Training Models for Biomedical Question Answering [article]

Sanjay Kamath, Brigitte Grau, Yue Ma
2019 arXiv   pre-print
Using deep learning models on small scale datasets would result in overfitting. To overcome this problem, the process of pre-training a model and fine-tuning it to the small scale dataset has been used extensively in domains such as image processing. Similarly for question answering, pre-training and fine-tuning can be done in several ways. Commonly reading comprehension models are used for pre-training, but we show that other types of pre-training can work better. We compare two pre-training
more » ... dels based on reading comprehension and open domain question answering models and determine the performance when fine-tuned and tested over BIOASQ question answering dataset. We find open domain question answering model to be a better fit for this task rather than reading comprehension model.
arXiv:1911.00712v1 fatcat:vsf5ur22ovbvvovvog55hq4lpq