Sang-won Leigh, Philipp Schoessler, Felix Heibeck, Pattie Maes, Hiroshi Ishii
2015 Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction - TEI '14  
The huge influx of mobile display devices is transforming computing into multi-device interaction, demanding a fluid mechanism for using multiple devices in synergy. In this paper, we present a novel interaction system that allows a collocated large display and a small handheld device to work together. The smartphone acts as a physical interface for near-surface interactions on a computer screen. Our system enables accurate position tracking of a smartphone placed on or over any screen by
more » ... ying a 2D color pattern that is captured using the smartphone's back-facing camera. As a result, the smartphone can directly interact with data displayed on the host computer, with precisely aligned visual feedback from both devices. The possible interactions are described and classified in a framework, which we exemplify on the basis of several implemented applications. Finally, we present a technical evaluation and describe how our system is unique compared to other existing near-surface interaction systems. The proposed technique can be implemented on existing devices without the need for additional hardware, promising immediate integration into existing systems.
doi:10.1145/2677199.2680584 dblp:conf/tei/LeighSHMI15 fatcat:sebx6jcpa5bvfmmac4zri46rg4