Visual Relationship Detection Based on Guided Proposals and Semantic Knowledge Distillation [article]

François Plesse, Alexandru Ginsca, Bertrand Delezoide, Françoise Prêteux
2018 arXiv   pre-print
A thorough comprehension of image content demands a complex grasp of the interactions that may occur in the natural world. One of the key issues is to describe the visual relationships between objects. When dealing with real world data, capturing these very diverse interactions is a difficult problem. It can be alleviated by incorporating common sense in a network. For this, we propose a framework that makes use of semantic knowledge and estimates the relevance of object pairs during both
more » ... ng and test phases. Extracted from precomputed models and training annotations, this information is distilled into the neural network dedicated to this task. Using this approach, we observe a significant improvement on all classes of Visual Genome, a challenging visual relationship dataset. A 68.5% relative gain on the recall at 100 is directly related to the relevance estimate and a 32.7% gain to the knowledge distillation.
arXiv:1805.10802v1 fatcat:3a6cqivkanbirpssu6hkw32aca