ImageSense: An Intelligent Collaborative Ideation Tool to Support Diverse Human-Computer Partnerships
Janin Koch, Nicolas Taffin, Michel Beaudouin-Lafon, Markku Laine, Andrés Lucero, Wendy E. Mackay
2020
Proceedings of the ACM on Human-Computer Interaction
Professional designers create mood boards to explore, visualize, and communicate hard-to-express ideas. We present ImageSense, an intelligent, collaborative ideation tool that combines individual and shared work spaces, as well as collaboration with multiple forms of intelligent agents. In the collection phase, ImageSense offers fluid transitions between serendipitous discovery of curated images via ImageCascade, combined text-and image-based Semantic search, and intelligent AI suggestions for
more »
... inding new images. For later composition and reflection, ImageSense provides semantic labels, generated color palettes, and multiple tag clouds to help communicate the intent of the mood board. A study of nine professional designers revealed nuances in designers' preferences for designer-led, system-led, and mixed-initiative approaches that evolve throughout the design process. We discuss the challenges in creating effective human-computer partnerships for creative activities, and suggest directions for future research. Janin Koch et al. tools should facilitate inspiration that stems from both convergent and divergent thinking, and help designers construct a new understanding of their work. Given recent advances in machine learning (ML) and artificial intelligence (AI), we are interested in how to incorporate intelligent assistance into the ideation process, while leaving the human designer in control. The key challenge is how to share agency. In traditional recommender systems, users provide initial input that an intelligent agent "aggregates and directs to appropriate recipients" [66] . Early 'mixed-initiative' approaches advocated user-centered agency, where users guide the system toward a desired goal [34] . More recent work suggests that users and machines can both have agency [6, 76] and should actively share it [41, 81] . As collaborative activities such as idea generation grow more complex, this form of shared agency becomes more appealing. However, this raises a key challenge: How can we design the interaction so that designers benefit from intelligent support, but still retain control? What does a satisfying and effective 'human-computer partnership' look like for complex, evolving and open-ended creative tasks such as mood board design? Furthermore, how can we accomplish this for highly collaborative tasks, where human designers, each with different ideation needs, collaborate with each other? We argue that effective tools should offer multiple forms of agency from entirely designer-led, to mixed, to system-led. This requires finding appropriate forms of interaction that support different levels of shared agency, between human designers and with the computer. We are interested in how to enhance the ideation workflow by increasing collaboration with other designers and providing multiple types of intelligent advice. We focus on mood boards, visual collages composed of images, text, and objects, that express concepts, ideas and emotions. Commonly used in creative fields such as design or fashion, they "stimulate the perception and interpretation of more ephemeral phenomena such as color, texture, form, image and status" [26] . Designers often collaborate in the design of physical mood boards, where the act of finding, choosing and curating visual material not only helps designers express ideas they already have, but also inspires new ideas based on their reactions to the images that emerge [36] . Mood boards let designers explore hard-to-express ideas [13] , and offer the potential for innovative discovery [26] . However, providing computational support for visual ideation is difficult, since designers' goals evolve rapidly. In this context, we pose the following research questions: RQ1: How can we integrate contributions from both human and intelligent agents seamlessly within a digital mood board? RQ2: Which kinds of intelligent assistance are appropriate for which types of ideation challenges? RQ3: How do human collaborators differ from intelligent assistants, and how can they support each other? In order to evaluate these questions we developed ImageSense, a digital mood board tool that supports both divergent and convergent thinking for visual ideation. ImageSense provides a collaborative environment for multiple designers to create a shared mood board, while taking advantage of several different intelligent tools that support different levels of inspiration and reflection. Specifically, designers can 1) select from a cascade of images curated by other designers, 2) search using text and images exploiting semantically enriched images, 3) take advantage of an intelligent agent that explores relevant images with the designer, and 4) receive reflection and sensemaking support. The key contributions of this paper include: 1) the design and implementation of ImageSense, an intelligent, collaborative digital mood board tool that supports the full ideation process, where human designers retain control of the interaction and actively choose the type and level of machine agency; and 2) a study with nine professional designers that assesses how ImageSense supports collaborative mood board design, and contrasts human-human and humanmachine collaboration in a realistic ideation task. We first discuss related work in creativity support tools, collaborative tools and intelligent assistants, and identify key implications for design. Next, we introduce ImageSense and illustrate it with a realistic use scenario. After describing the technical details, we report the results of the study,
doi:10.1145/3392850
fatcat:slgz3cg4mbg7thhquphjy5tnsy