A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf
.
Beyond Accuracy: A Consolidated Tool for Visual Question Answering Benchmarking
[article]
2021
arXiv
pre-print
On the way towards general Visual Question Answering (VQA) systems that are able to answer arbitrary questions, the need arises for evaluation beyond single-metric leaderboards for specific datasets. To this end, we propose a browser-based benchmarking tool for researchers and challenge organizers, with an API for easy integration of new models and datasets to keep up with the fast-changing landscape of VQA. Our tool helps test generalization capabilities of models across multiple datasets,
arXiv:2110.05159v1
fatcat:hhjfdvhpgzbunfgnq3h5qdzjxq