Reconstruction of Pilot Behaviour from Cockpit Image Recorder

Hiroka Tsuda, Olaf Stroosma, Max Mulder
2020 AIAA Scitech 2020 Forum   unpublished
A method to automatically identify pilot actions from cockpit camera footage is reported in this paper. Although they have long been considered for the enhancement of flight safety, cockpit image recorders have not yet been standard equipment in aircraft cockpits. The rules on Flight Data Recorders have been changed, however, to include a cockpit image recorder as one of the safety devices, and it is recommended to be installed in small aircraft as a substitute for a Flight Data Recorder. With
more » ... ockpit images becoming available, it would surely be useful for accident analysis as well as for daily flight analysis. Especially for the latter purpose, pilot behavior should be automatically analyzed and classified into specific actions, or procedures. The authors conducted a study to assess the feasibility of automatic detection of pilot actions in the cockpit by a machine learning process. Results show that even with a small amount of training data, the resulting algorithm could identify some typical actions, such as manipulation of the switches on the glare shield, with 80% accuracy. Even in cases with a button and a switch positioned very close to each other, the actions 'pushing the switch' and 'pushing the button' could be distinguished by the algorithm. The action estimation accuracy improves up to 90% when using the training data focused on the pilot's body parts, rather than the data focused on the whole body. Downloaded by TU DELFT on January 9, 2020 | http://arc.aiaa.org |
doi:10.2514/6.2020-1873 fatcat:rpcfy4sbxndrjk6boofa5xxsn4