Internet Archive Scholar

The 2013 SESAME Multimedia Event Detection and Recounting System

Robert C. Bolles, J. Brian Burns, James A. Herson, Gregory K. Myers, Stephanie Pancoast, Julien van Hout, Wen Wang, Julie Wong, Eric Yeh, AmirHossein Habibian, Dennis C. Koelma, Zhenyang Li (+9 others)
2013 TREC Video Retrieval Evaluation  

Preserved Fulltext

fulltext thumbnail
Web Archive Capture PDF (408.5 kB)
https://web.archive.org/web/20210829171834/https://www-nlpir.nist.gov/projects/tvpubs/tv13.papers/sesame.pdf

A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL. The file type is application/pdf.

The SESAME team submitted runs as a full participant in the MED13 evaluation, and submitted video, motion, and audio features; high-level semantic concepts for visual objects, scenes, persons, and actions; automatic speech recognition (ASR); and video optical character recognition (OCR). The individual types of features and concepts produced a total of eight event classifiers. We combined the event detection results for these classifiers using arithmetic mean and log-likelihood ratio fusion
more » ... ods, and developed and applied a method for selecting the detection threshold. The SESAME system generated event recountings by selecting intervals based on the semantic concepts, and on concepts recognized by ASR and OCR. Our major findings are:  Our strategy of first selecting the most informative interval for a video, and then determining the most appropriate event-related semantic concepts within that interval to display for multimedia event recounting (MER), produced the best ObsTextScore in the evaluation. (The ObsTextScore measures the judges' responses to the question "How well does the text of this observation describe the snippet(s)?".)  The multimedia event detection (MED) performance for 100Ex and 10Ex was dominated by the classifiers that exploited visual content.  The ASR and OCR classifiers for 0Ex performed better than those trained with 10Ex.  The log-likelihood ratio late-fusion method demonstrated improved performance over simple averaging of event detection scores for 100Ex, but not for 10Ex.
dblp:conf/trecvid/BollesBHMPHWWYH13 fatcat:a75734gfbvddlgwijke7qt62nm
Citation

Robert C. Bolles, J. Brian Burns, James A. Herson, Gregory K. Myers, Stephanie Pancoast, Julien van Hout, Wen Wang, Julie Wong, Eric Yeh, AmirHossein Habibian, Dennis C. Koelma, Zhenyang Li, Masoud Mazloom, Silvia-Laura Pintea, Arnold W. M. Smeulders, Cees G. M. Snoek, Sung Chun Lee, Ram Nevatia, Pramod Sharma, Chen Sun, Rémi Trichet. "The 2013 SESAME Multimedia Event Detection and Recounting System." TREC Video Retrieval Evaluation (2013)