Humans possess only five senses and very effectively coordinate their cross-sensory perceptions to situate themselves in uncertain operational environments for extracting context relevant actionable intelligence. Machines, on the other hand, may be embodied with a wider variety of electronic sensing devices but lack such situational intelligence in interpreting the sensed information. Despite significant advances in sensing technologies, machine perception remains primitive when compared to

more »

... n perception. Lack of situational intelligence results in processing of large amounts of irrelevant information, leading to the often cited "curse of dimensionality" and computational explosions. These, in turn, limit the power of datadriven abstract reasoning and problemsolving algorithms, cause a lack of focus for drawing upon relevant past knowledge, and inhibit situational learning. As a consequence, autonomous systems cannot be trusted to adapt their behavior to unanticipated operational conditions. Current behavior-based modeling approaches address these issues by developing world models that modularly decompose the problem space. This requires a very detailed and somewhat complete understanding of the operational environments as a prerequisite. Yet, such models invariably prove inadequate for real world operations due to the rigidness of the decompositions. Autonomous system designs, therefore, are not robust and machine learning methods remain brittle. Situationally aware sensor fusion and machine perception present a new frontier in machine automation, which holds the promise of unprecedented levels of autonomy in executing complex tasks in dynamic operational environments. The goal of such automation is to accomplish these tasks with the perception and adaptation of humans, and often in collaboration with humans. Several technological challenges must be addressed to further the state-of-the-art toward this goal. CONTEXT LEARNING AND IN SITU DECISION ADAPTATION Faced with the challenge of data to action in a complex noisy world, research methods have emerged in diverse fields, over the past decade, for machines to extract current operational context from sensor data. These include physics based environmentally adaptive sensing models, innovations in image and scene processing, natural language processing, ubiquitous computing, and cognitive neuroscience. Current state-of-the-art research in these areas attempts to extract operational context for a specific sensing modality, like visual or auditory context, which is not relevant to other modalities. The notion of context itself is often incoherent and ill-defined across sensing modalities and applications: image processing research generally assumes only the visual scene to be the context for object detection; for human-machine interactions, context is often the linguistic semantics in which humans express the current instruction to autonomous systems; for ubiquitous or mobile computing, it is the computing environment, and in cognitive sciences, context is often modeled via attention and memory. In a multi-sensor operational environment, involving both hard and soft sensing modalities, a broad unified notion of context is needed. This notion should characterize situations in the physical, electronic, or tactical environments that affect the acquisition and interpretation of heterogeneous sensor data for machine perception and adaptation to specified goals. Furthermore, it is often necessary to iteratively sense the context automatically and treat it as an implicit dynamic input to the application for robust context-aware operations. In their 2013 paper, Blasch et al. (2013) survey recent research efforts to accommodate the effects of context in information fusion for target tracking applications. Several of these approaches attempt to mitigate the effects of context on the feature space by designing statistical detection and classification algorithms that are invariant to context changes. However, feature extraction techniques often do not adapt well to the highly non-linear and nonstationary effects of the operational environment. An alternate approach to improving detection performance is to exploit differences in sensor behaviors across environments and treat them as a supplemental source for context-dependent-learning. This approach was recently proposed in Frigui et al. (2010) for learning regions of similar responses for each sensor. Formalizing this approach, a mathematical characterization of machine extractable context, applicable to all sensing modalities relevant to an application, was recently presented in Phoha et al. (2014) , with the objective of enabling contextual decisionmaking in dynamic data-driven classification systems. Both intrinsic context, i.e., factors, which directly affect sensor measurements, as well as extrinsic context, i.e., factors that do not affect sensor measurements directly, but affect the interpretation of observed data, were analytically formulated. This analytical foundation can be used to characterize and represent situational intelligence for multi-sensor multitarget applications. Further work in integrating data-driven and model based methods for context learning, discovery of new www.frontiersin.org