Assessment of Nontechnical Skills

Wolff-Michael Roth, Timothy J. Mavin
2013 Aviation Psychology and Applied Human Factors  
The assessment of pilots' performances and more specifically their non-technical skills turns out to be difficult because researchers and practitioners find that interrater reliability tends to be moderate to low. Some researchers even suggest abandoning non-technical skills currently in use, including situational awareness, because they correspond to folk psychology and have no scientific basis. In this study, we propose a different tack to the issue of variance in pilot performance
more » ... formance assessment. Observations in the field show that assessment is a categorization rather than a measurement issue. Categorization, even though and especially when based on imprecise information and assessments, can be modeled mathematically using fuzzy logic. That is, the variance in performance ratings is not unexplained variance, as assumed in the measurement model, but explainable variation. We present the structure of the approach and provide an example from a large database of pilots providing reasons for their assessments of other pilots' (simulator) flying performances. As implications, we discuss possibilities for using the variations in performance ratings positively, as a source for learning and resilience building.
doi:10.1027/2192-0923/a000045 fatcat:s4czfhs4avfx5dsmy3moel36me