Multimodal cueing and assessment of pilot engagement during low level flight

Low-level flight missions can be complex, at times requiring any or all of the following: maneuvering and navigating over challenging terrain, scanning for hostile or friendly units, operation in degraded visibility, radio communications, and decision-making in uncertain and dynamics environments. These conditions, and time, will affect mental workload (MWL) and performance. While direct performance measurements are normally available, information on pilot workload must either be obtained through (intrusive) subjective measures directly from the pilot, or inferred using indirect measurements. Performance can affect MWL, and MWL can affect performance - as the pilot generally perceives and responds to task performance through display interfaces, these displays can be used to manage and balance the tradeoff between MWL and performance. This paper presents the work of a collaborative project between US and Israel whose objectives are to develop a multimodal integrated cueing environment for near-earth helicopter operations, and to validate measures for assessing pilot workload for real-time and post-mission applications. A first simulation experiment was conducted that examined 1) visual cueing depicting both predicted terrain slope and aircraft height-above-ground, and 2) spatial (3D) auditory cueing for depicting predicted deviation from desired height-above-ground and impending collision with terrain. Collected Measures included altitude error, control rate and Bedford rating. Initial results indicate that synergistic visual and auditory cueing can enhance performance, therefore could be used to reduce pilot workload while sustaining performance. A second simulation experiment was designed to assess MWL and task engagement under different workload levels encountered during a low-level mission that included the following conditions: unlimited vs. degraded visibility, presence or absence of obstacles and/or targets and terrain difficulty. Two local measures of workload were favored over a global measure: (1) oculomotor behavior, including the Index of Cognitive Activity (ICA) and (2) pilot's momentary behavior (micro-performance). Preliminary results show promise for using some of these measures as real-time indicators of pilot workload and engagement. The results of these two experiments will provide a framework for the development and evaluation of future workload-adaptive multimodal display concepts for helicopter operations during low-level flight.