Cross-Device Taxonomy

Frederik Brudy, Christian Holz, Roman Rädle, Chi-Jui Wu, Steven Houben, Clemens Nylandsted Klokmose, Nicolai Marquardt
2019 Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19  
Designing interfaces or applications that move beyond the bounds of a single device screen enables new ways to engage with digital content. Research addressing the opportunities and challenges of interactions with multiple devices in concert is of continued focus in HCI research. To inform the future research agenda of this field, we contribute an analysis and taxonomy of a corpus of 510 papers in the crossdevice computing domain. For both new and experienced researchers in the field we
more » ... an overview, historic trends and unified terminology of cross-device research; discussion of major and under-explored application areas; mapping of enabling technologies; synthesis of key interaction techniques spanning across multiple devices; and review of common evaluation strategies. We close with a discussion of open issues. Our taxonomy aims to create a unified terminology and common understanding for researchers in order to facilitate and stimulate future cross-device research. Mirrored Second screen Asynchronous (Sequential use) vs. iOS Android Migratory interfaces Description: adaptive interfaces that are capable to migrate between multiple devices (in particular, different form factors). Related taxonomy: Reference model [76] Description: development of interfaces that run on different device form factors and operating system platforms. A A 1 2 3 Distributed UI Logical distribution Spatial distribution Description: logical distribution of distinct UIs or elements across a number of devices. Related taxonomy: [86] Description: spatial distribution of the continuous visible area of one screen across a number of devices. Related taxonomies: [276, 292, 330] Description: mirroring the exact same content on multiple screens/displays. Pair, select or engage with device by: → stitching [54, 129] → pinching [190, 192, 238, 243] → synchronous tapping [280] → vision-based handshake [360] → pointing [259, 349] → finger postures [141] → painting on surfaces [370] Pair, select or engage with device by: → touching the surface [293] → knocking or tapping [103, 280] → tapping appliance [372] → taking picture to recognize device [299] → roll-and-pointing [61] → hold-and-flipping [54] → pitching-to-open [54] Pair or engage with device by: → bumping [127] → synchronous gestures [272] → stacking [61] → snapping a picture pair [60] → shaking [131, 211] → placing down device [143, 270, 360] → recognizing motion correlation [359] Pair or engage with device by: → bending [61] → using sandwich structure [61] → stretching [61] Pair or engage with device by: → approaching [12, 203] → turning body towards [12, 107, 205, 369] → detecting presence of person [12, 30, 145, 369] → detect groups [30, 206] → detect head position [110] → perspective awareness [226] Transfer content by: → dragging [312] → Pick-and-drop [223, 278] → swiping [159, 253, 348] → tapping [310] → flicking [253, 277] → SuperFlick [277] → HyperDrag [281] → pinch, swing [253] → corresponding gestures [223] → pantograph directing [223] → slingshot targeting [223] Interact with content by: → changing finger posture [141] → dragging on proxies or portals [64, 123, 203, 206] → panning [270] → press-and-flick [223] → cross-dev. pinch-to-zoom [206] → drag-and-pop [15] → drag-and-pick [15] Explore content with: → broadcast contextual cues, spatially-agnostic [116] → display pointers [325] → swipe on watch [54, 141] Transfer content by: → touching the surface with edge of device [294] → dragging content in negative space between devices [143] → pointing with phone, touch and drag, release touch to stop [30] → waving between [50] → waving above [141, 270] → point-and-grab [198] → lift-and-drop [8] → grasp and micro-mobility: fine-grained reference, hold to refer back [206] → drawing a line on the surface between devices [103]
doi:10.1145/3290605.3300792 dblp:conf/chi/BrudyHRWHKM19 fatcat:xeo4p3w4kjgghfe6yvug5bahm4