EXPLOITING MULTISPECTRAL AND CONTEXTUAL INFORMATION TO IMPROVE HUMAN DETECTION Exploiting Multispectral and Contextual Information to Improve Human Detection

Jingjing Liu, Jingjing Liu, Dimitris Metaxas
Human detection has various applications, e.g., autonomous driving car, surveillance system, and retail. In this dissertation, we first exploit multispectral images (i.e., RGB and thermal images) for human detection. We extensively analyze Faster R-CNN for the detection task and then model multispectral human detection into a fusion problem of convolutional networks (ConvNets). We design four distinct ConvNet fusion architectures that integrate two-branch ConvNets on different stages of neural
more » ... t stages of neural networks, all of which yield better performance compared with the baseline detector. In the second part of this dissertation, we leverage instance-level contextual information in crowded scenes to boost performance of human detection. Based on a context graph that incorporates both geometric and social contextual patterns from crowds, we apply progressive potential propagation algorithm to discover weak detections that are contextually compatible with true detections while suppressing irrelevant false alarms. The method significantly improves the performance of any shallow human detectors, obtaining comparable results to deep learning based methods. ii Acknowledgements First of all, I would like to express my deepest gratitude to my advisor, Prof. Dimitris N. Metaxas, for his generous support and guidance throughout the past six years. Prof. Metaxas not only continuously encourages me to solve challenging and practical problems but also provides me with the excellent research environment that I have sufficient research freedom and lots of opportunities to collaborate with world-class researchers. He always gives me beneficial pieces of advice whenever I feel confused in research or life. His great passion in research, remarkable leadership, wealthy knowledge, and unexhausted energy set him an extraordinary example for me in both life and career.