Models of attention in computing and communication
Communications of the ACM
Introduction One of the main results of Twentieth-century Cognitive Psychology is that, despite the overall impressive abilities of people to sense, remember, and reason about the world, our cognitive abilities are extremely limited in well-characterized ways. In particular, psychologists have found that people grapple with scarce attentional resources and limited working memory. Such limitations become salient when people are challenged with remembering more than a handful of new ideas or
... in the short term [20, 28] , recognizing important targets against a background pattern of items [5, 26] , or interleaving multiple tasks [6, 26] . These results indicate that we cannot help but to inspect the world via a limited spotlight of attention. As such, we often generate clues implicitly and explicitly about what we are selectively attending to and how deeply we are focusing. Given constraints on attentional resources, it is no surprise that communication among people relies deeply on attentional signals. Psychologists and linguists studying communication have recognized that signaling and detecting attentional states lies at the heart of the fast-paced and fluid interactions that people have with one another when collaborating or communicating [2, 7] . Attentional cues are central in decisions about when to initiate or to make an effective contribution to a conversation or project. Beyond knowing when to speak or listen in a conversation, attention is critical in detecting that a conversation is progressing. More generally, detecting or inferring attention is an essential component of the overall process of grounding-converging in a shared manner on a mutual understanding of a communication  . The findings about our limited attentional resources-and about how we rely on attentional signals in collaborating-have significant implications for how we design computational systems and interfaces. Over the last five years, our team at Microsoft Research has explored, within the Attentional User Interface (AUI) project, opportunities for enhancing computing and communications systems by treating human attention as a central construct and organizing principle. We consider attention as a rare commodity-and critical currency-in reasoning about the information awareness versus disruption of users  . We have also pursued the use of attentional cues as an important source of rich signals forms of fluid mixed-initiative collaborations with computers  . Moving to considerations of computational efficiency, an assessment of a user's current and future attention can be employed to triage computational resources. Investigations in this realm include selective allocation of resources in rendering graphics via relying on models [14, 16] or on direct observations  of visual attention, and in guiding precomputation and prefetching  with forecasts of future attention. Finally, although there is a rich history of prior work on attention from cognitive psychology, we have found that there is much we do not yet understand. Thus, beyond pooling results from prior psychological studies, we need to continue to perform user studies that adapt or extend prior results on attention and memory from cognitive psychology to real-world computing and communication applications [3, 4, 18, 19]. We shall first describe several principles and methodologies at the heart of research on integrating models of attention into human-computer interaction and communications. Then, we shall review representative efforts that illustrate how we can harness these principles in attention-sensitive messaging and mixed-initiative interaction applications.