Tagging over time

Ritendra Datta, Dhiraj Joshi, Jia Li, James Z. Wang
2007 Proceedings of the 15th international conference on Multimedia - MULTIMEDIA '07  
Automatic image annotation has been a hot-pursuit among multimedia researchers of late. Modest performance guarantees and limited adaptability often restrict its applicability to real-world settings. We propose tagging over time (T/T) to push the technology toward real-world applicability. Of particular interest are online systems that receive user-provided images and feedback over time, with user focus possibly changing and evolving. The T/T framework consists of a principled probabilistic
more » ... oach to metalearning, which acts as a go-between for a 'black-box' annotation system and the users. Inspired by inductive transfer, the approach attempts to harness available information, including the black-box model's performance, the image representations, and the WordNet ontology. Being computationally 'lightweight', this meta-learner efficiently re-trains over time, to improve and/or adapt to changes. The black-box annotation model is not required to be re-trained, allowing computationally intensive algorithms to be used. We experiment with standard image datasets and real-world data streams, using two existing annotation systems as blackboxes. Both batch and online annotation settings are experimented with. It is observed that the addition of this metalearning layer produces much improved results that outperform best-known results. For the online setting, the T/T approach produces progressively better annotation with time, significantly outperforming the black-box as well as the static form of the meta-learner, on real-world data.
doi:10.1145/1291233.1291328 dblp:conf/mm/DattaJLW07 fatcat:76ad6sovvjcgxk5hoayujhfwnm