Tracking Multiple Colored Blobs with a Moving Camera

A.A. Argyros, M.I.A. Lourakis
2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)  
DESCRIPTION This work concerns a method for tracking multiple blobs exhibiting certain color distributions in images acquired by a possibly moving camera. The method encompasses a collection of techniques that enable modeling and detecting the blobs possessing the desired color distribution(s), as well as infering their temporal association across image sequences. Appropriately colored blobs are detected with a Bayesian classifier which is bootstrapped with a small set of training data. Then,
more » ... ining data. Then, an on-line iterative training procedure is employed to refine the classifier using additional training images. Online adaptation of color probabilities is used to enable the classifier to cope with illumination changes. Tracking over time is realized through a novel technique which can handle multiple colored blobs. Such blobs may move in complex trajectories and occlude each other in the field of view of a possibly moving camera, while their number may vary over time. A prototype implementation of the developed system running on a conventional Pentium IV processor at 2.5 GHz operates on 320x240 live video in real time (30Hz). It is worth pointing out that currently, the cycle time of the tracker is determined by the maximum acquisition frame rate that is supported by our IEEE 1394 camera, rather than the latency introduced by the computational overhead for tracking blobs. Briefly, the developed method operates as follows. At each time instant, the camera acquires an image on which the appropriately colored blobs (i.e. connected sets of colored pixels) are detected. A set of blob hypotheses that have been tracked up to the current time instant is also being maintained. The detected blobs, together with the blob hypotheses are then associated in time. The goal of this association is, on the one hand, to assign a new, unique label to each new blob that enters the camera's field of view for the first time, and on the other hand, to propagate in time the labels of already detected blobs. More details regarding the approach adopted towards solving the aforementioned subproblems can be found in [1] . The ability of the proposed tracker to adapt to any desired color distribution through training, makes it a building block that is well-suited to various applications that are in need of efficient and robust color tracking that is tolerant to illumination changes. For instance, the proposed method has been employed in tracking multiple skin-colored regions such as human hands and faces [1] . Simple gesture recognition techniques applied to the output of the skin-colored regions tracker have resulted in a system that permits a human to remotely control a computer, substituting the mouse with his/her hands. These gesture recognition techniques are based on detecting the fingers in skin-colored regions corresponding to human hands. By employing a binocular camera system and two distinct instances of the skin-color tracker, each operating on a separate video stream, we have also developed a method which is able to recover the 3D position of all skin-colored blobs that are visible in both streams [2] . Another application of the developed tracker concerns its use for tracking color blobs that function as landmarks in angle-based robot navigation [3] . The accompanying video demonstrates the use of the proposed blob tracker for tracking skin-colored objects. The video shows the contours of the tracked skin-color objects superimposed on the original sequence, using a different color for each distinct object. All processing was done in real-time, in parallel with image acquisition. As can be clearly seen, the tracker can successfully track the face and hands of a man, despite them undergoing complex motions, changing their shape, overlapping each other and leaving/re-entering the field of view. Furthermore, the video demonstrates finger detection based on multi-scale processing of the contours pertaining to skin-colored objects. ACKNOWLEDGEMENT
doi:10.1109/cvpr.2005.348 dblp:conf/cvpr/ArgyrosL05 fatcat:yam5tvhvt5abtblq5v6eywsyu4