Multi-Modal Word Synset Induction

Jesse Thomason, Raymond J. Mooney
2017 Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence  
A word in natural language can be polysemous, having multiple meanings, as well as synonymous, meaning the same thing as other words. Word sense induction attempts to find the senses of polysemous words. Synonymy detection attempts to find when two words are interchangeable. We combine these tasks, first inducing word senses and then detecting similar senses to form word-sense synonym sets (synsets) in an unsupervised fashion. Given pairs of images and text with noun phrase labels, we perform
more » ... nset induction to produce collections of underlying concepts described by one or more noun phrases. We find that considering multi-modal features from both visual and textual context yields better induced synsets than using either context alone. Human evaluations show that our unsupervised, multi-modally induced synsets are comparable in quality to annotation-assisted ImageNet synsets, achieving about 84% of ImageNet synsets' approval.
doi:10.24963/ijcai.2017/575 dblp:conf/ijcai/ThomasonM17 fatcat:xea3e5jodramlfq5tchotqwxba