Real-Time Nonintrusive Monitoring and Prediction of Driver Fatigue

Q. Ji, Z. Zhu, P. Lan
2004 IEEE Transactions on Vehicular Technology  
This paper describes a real-time prototype computer vision system for monitoring driver vigilance. It uses a remotely located CCD camera equipped with an active IR illuminator to acquire video images of the driver. Various visual cues typically characterizing the level of alertness of a person are extracted in real time and systematically combined to infer the fatigue level of the driver. The visual cues used include eyelid movement, gaze movement, head movement, and facial expression. A
more » ... listic model is developed to model human fatigue and to predict fatigue based on the visual cues obtained. The simultaneous use of multiple visual cues and their systematic combination yields a much more robust and accurate fatigue characterization than using a single visual cue. The system was validated under real life fatigue conditions with human subjects of different ethnic backgrounds, different genders, ages, with/without glasses, and under different illumination conditions, and it was found reasonably robust, reliable and accurate in fatigue characterization. • In-vehicle, on-line, operator status monitoring technologies The technologies in this category seek to real-time record some bio-behavioral dimension(s) of an operator, such as feature of the eyes, face, head, heart, brain activity, reaction time etc., during driving [19] , [20] , [21] . According to the different methods used for measurements, the technologies can be further divided into three types. The first type employs electroencephalograph measures (EEG), based on which most of successful equipments developed are off-line versions. Also, there is an on-line version called "Mind Switch" that uses a headband device, in which, the electrodes are embedded to make contact with the driver's scalp so as to measure the brain waves. Ocular measures are used in the second type, which is considered as the most suitable way for on-line monitoring. So far, many eye blinking, pupil response, eye closure and eye movement monitors have been developed. Other physiological/bio-behavioral measures used in the third type include tone of facial muscles (facial expression), body postures and head noddings. Among different techniques, the best detection accuracy is achieved with techniques that measure physiological conditions like brain waves, heart rate, and pulse rate [9], [23] . Requiring physical contact with drivers (e.g., attaching electrodes) to perform, these techniques are intrusive, causing annoyance to drivers. Good results have also been reported with techniques that monitor eyelid movement and eye gaze with a head-mounted eye tracker or special contact lens. Results from monitoring head movement [24] with a headmount device are also encouraging. These techniques, though less intrusive, are still not practically acceptable. A driver's state of vigilance can also be characterized by the behaviors of the vehicle he/she operates. Vehicle behaviors including speed, lateral position, turning angle, and moving course are good indicators of a driver's alertness level. While these techniques may be implemented non-intrusively, they are, nevertheless, subject to several limitations including the vehicle type, driver experiences, and driving conditions [3] . People in fatigue exhibit certain visual behaviors easily observable from changes in facial features like the eyes, head, and face. Visual behaviors that typically reflect a person's level of fatigue include eyelid movement, gaze, head movement and facial expression. To make use of these visual cues, another increasingly popular and non-invasive approach
doi:10.1109/tvt.2004.830974 fatcat:ovdk2ol2yfaafnxuu75p5lrhei