Interaction Design for Complex Cognitive Activities with Visual Representations: A Pattern-Based Approach
AIS Transactions on Human-Computer Interaction
This paper is concerned with interaction design for visualization-based computational tools that support the performance of complex cognitive activities, such as analytical reasoning, sense making, decision making, problem solving, learning, planning, and knowledge discovery. In this paper, a number of foundational concepts related to interaction and complex cognitive activities are syncretized into a coherent theoretical framework. This framework is general, in the sense that it is applicable
... o all technologies, platforms, tools, users, activities, and visual representations. Included in the framework is a catalog of 32 fundamental epistemic action patterns, with each action pattern being characterized and examined in terms of its utility in supporting different complex cognitive activities. This catalog of action patterns is comprehensive, covering a broad range of interactions that are performed by a diverse group of users for all kinds of tasks and activities. The presented framework is also generative, in that it can stimulate creativity and innovation in research and design for a number of domains and disciplines, including data and information visualization, visual analytics, digital libraries, health informatics, learning sciences and technologies, personal information management, decision support, information systems, and knowledge management. the design of visual representations, 2) a framework dealing with the analysis of the ontological properties of visual representations that affect the performance of complex cognitive activities, and 3) a framework dealing with the detailed analysis of the anatomical structure of an individual interaction as well as the manner in which interactions are combined and integrated during the performance of complex cognitive activities. The component of EDIFICE that is developed in this paper deals with the interaction design of CASTs. The interaction that takes place between a user and a CAST can be characterized at multiple levels of granularity (see Sedig et al., 2013) . Such levels include macrolevel activities, tasks, individual actions and reactions (i.e., interactions), and micro-level events. Even though this paper discusses interaction at all these levels, its focus is mainly on interaction at the level of individual actions and reactions, dealing primarily with pattern-based characterizations of actions and their utility in supporting complex cognitive activities. Because this framework is human-centered, focusing on the action component of interaction, and because it takes a pattern-based approach, we will henceforth refer to it as EDIFICE-AP (where AP stands for Action Patterns). The rest of this paper is divided into 3 main sections: 1) conceptual and theoretical foundations; 2) the EDIFICE-AP framework; and 3) summary and future directions. CONCEPTUAL AND THEORETICAL FOUNDATIONS This section serves a twofold function. First, it examines the utility of, and need for, frameworks; second, it identifies and explicates a number of terms and concepts that are necessary for discussing human-information interaction in the context of CASTs. These terms and concepts have been used in different contexts often with different meanings and connotations. It is necessary, therefore, to characterize them and examine their relationships before presenting the EDIFICE-AP framework. Frameworks Since the time of Plato and Aristotle, frameworks and classification systems have played an important role in systematic and scientific exploration of phenomena (Darian, 2003) . Conducting research to develop frameworks, taxonomies, and models is crucial to the advancement of any discipline, including the analysis and design of computational tools (Carroll, 1991; Heller et al. 2001; Hult et al., 2006) . Bederson and Shneiderman (2003) enumerate five roles for this type of research: 1) describe (characterize objects and actions in a systematic manner to provide clear language and enable cooperation), 2) explain (explain processes to support education), 3) predict (predict performance in different situations), 4) prescribe (suggest guidelines and best practices), and 5) generate (facilitate innovation). In the case of interaction design for CASTs, a framework concerned with epistemic action patterns can serve each of these roles. Moreover, by classifying the space of potential actions and characterizing these actions, a catalog of action patterns can provide a common language for referring to potential actions, and can provide opportunities for their systematic analysis and comparison. As we try to design CASTs that require attentive, mindful engagement with information, some researchers are highlighting the importance of careful study of the transactions that users make as they interact with these tools (e.g., Kim and Reeves, 2007; Brey, 2005; Dascal and Dror, 2005; Thomas and Cook, 2005) . A framework such as EDIFICE-AP can provide investigators with a systematic support structure for thinking about these transactions. Without frameworks that organize and characterize fundamental aspects of the interaction design space, the approach to both research and practice must be largely adhoc and rely mostly on personal anecdotes and intuition. Information and Information Space Information can originate from many different sources (Bates, 2006) . These sources can be concrete (e.g., a molecule), existing within a physical space, or abstract (e.g., financial markets), originating from a non-tangible, nonperceptible source. An information space is an environment, source, domain, place, or area of containment from which a body of information originates. The concept of information does not yet have a universally agreed-upon definition, and is defined in different ways depending on the context in which it is used (Marchionini, 2010). We adopt Bates ' (2005, 2006) definition of information-that information is the pattern of organization of matter and energye.g., physical objects, energy fields and forces, conceptual structures, and semantic relationships. This definition of information is broad and encompasses all visible, invisible, concrete, and abstract organizational patterns and sources-micro entities (e.g., DNA structure of a cell), hard-to-reach entities (e.g., rocks on distant planets), and nonphysical entities (e.g., scientific concepts). Information by itself does not have inherent meaning. Meaning must be assigned to it (Stonier, 1990) . For instance, electromagnetic waves travelling through space have no meaning until they are interpreted in a contextual setting. As such, giving meaning to information and integrating it into other preexisting mental forms is an essential feature of any complex cognitive activity (Bates, 2005; Sternberg and Ben-Zeev, 2001) . He has been doing research in the area of human-centered interactive visualizations since 1993. He is interested in the design of computer-based tools that help people perform information-intensive complex cognitive activities, such as sense making, decision making, data analysis, and learning. As such, his research and publications span a range of topics such as data and information visualization, visual analytics, human-information interaction design, information interface design, medical and health informatics, digital cognitive games, and cognitive and learning technologies. In the past few years, he has been working on the development of comprehensive frameworks that make the design and evaluation of visualizations and interactions more scientific. Paul Parsons is a PhD candidate in computer science at Western University, Canada. His research explores technology-mediated human-information interaction to understand how interactive technologies can facilitate and enhance thinking and reasoning processes, with a particular focus on complex cognitive activities such as sense making, decision making, learning, and analytical reasoning. Application areas of his research include data and information visualization, visual analytics, information systems, medical and health informatics, and educational and cognitive technologies.