Query-by-Emoji Video Search

Spencer Cappallo, Thomas Mensink, Cees G.M. Snoek
2015 Proceedings of the 23rd ACM international conference on Multimedia - MM '15  
This technical demo presents Emoji2Video, a query-by-emoji interface for exploring video collections. Ideogram-based video search and representation presents an opportunity for an intuitive, visual interface and concise non-textual summary of video contents, in a form factor that is ideal for small screens. The demo allows users to build search strings comprised of ideograms which are used to query a large dataset of YouTube videos. The system returns a list of the top-ranking videos for the
more » ... r query along with an emoji summary of the video contents so that users may make an informed decision whether to view a video or refine their search terms. The ranking of the videos is done in a zero-shot, multi-modal manner that employs an embedding space to exploit semantic relationships between user-selected ideograms and the video's visual and textual content.
doi:10.1145/2733373.2807961 dblp:conf/mm/CappalloMS15 fatcat:aawwfdkkpvdivbmx5u6smzpuhu