A flick in the right direction: a case study of gestural input

M Moyle, A Cockburn
2005 Behavior and Information Technology  
This paper describes the design and evaluation of a gesture-based scheme for issuing the back and forward commands in web browsers. In designing our gesture recogniser we conducted several experiments to determine metrics associated with the magnitude, timing and stereotypical errors of 'natural' linear flick gestures using stylus and mouse input devices. These lowlevel metrics are important to software designers who must implement algorithms that discriminate between gestures and other actions
more » ... such as mouse clicks and drags. As well as empirically characterising gestures, the metrics provide various insights into stereotypical behaviour with gestures, including the facts that angular errors are larger in the left and right directions with the pen, that vertical gestures are 'awkward' with the mouse, and that downwards gestures are slower than other directions. An evaluation of gestures for web browsing shortcuts shows that they enhance navigation efficiency, and that participants were extremely enthusiastic about them. This section reviews related work on gesture-based interfaces. Gesture input is used to control a wide range of user interfaces, from simple mouse-based flicks in marking menus (see below), through to rich free-form hand and body gestures used to control and interact within 3D virtual worlds-Wexelblat (1995), for instance, describes a system for gesture control of a virtual environment, and Paradiso (2003) describes recent work on using free-form gestures to interact with a large displays. This review focuses on gestures created using standard mouse and stylus input devices. The section also describes related work on web navigation, which is our target application domain for gestural input. 1 www.opera.com. Gesture Systems Mouse-based gesture systems parse the direction or rate of mouse movement, and map the motion onto a desired user action. One of the first systems to feature mouse-gestures was Broderbund's 'Shufflepuck Café ' (released in 1989), in which the user slides a puck towards an opponent with the puck's speed and direction determined by a rapid mouse-drag gesture. Kurtenback and Buxton (1991) were among the first to investigate mouse gestures for issuing everyday commands in user interfaces through their 'marking menus', which were a specialisation of pie-menus (Callahan, Hopkins, Weiser and Shneiderman 1988). Pie-menus minimise item selection time by arranging menu items around a circle centred on the user's cursor. The Fitts' Law (1954) time-to-target requirements are minimised because a movement of one-pixel is sufficient to reach any of the menu items, and further movements result in the target effectively becoming larger. Marking menus enhance the efficiency of pie-menus by allowing the user to select items simply by flicking the mouse in the appropriate directionthe user need not wait for the menu to be displayed. If the user hesitates during the gesture (a delay of more than approximately half a second) then the pie-menu is displayed to assist learning the gesture set. Kurtenbach and Buxton (1994) found that marking menus were heavily used once users learned the gesture direction. Further evaluation showed that performance with marking menus deteriorates as the number of items in the menu increases (Kurtenbach, Sellen and Buxton 1993). Early mouse-based marking menus systems used the left mouse button to issue gesture commands. The left button, however, is often used for other tasks such as text selection. For this reason, some systems (such as Opera, see section 2.2) use the right mouse button to reduce the problems of overloading the interface semantics associated with each button, yet users are unfamiliar with dragging actions using the right button. The evaluations reported in this paper include an examination of the differences between gestures that are created using the left and right mouse-buttons. In selecting one item from a single marking menu, the recognition software need only compare the total distance travelled on the X and Y coordinates to determine the direction of the gesture. By extending the marking menu concept to non-linear gestures, users can access much larger command sets, but the recognition software needs greater sophistication to distinguish between differently shaped gestures. The Unistrokes gesture alphabet, for example, allows users to express all letters in the Roman alphabet with gestures (Goldberg and Richardson 1993) . Several other character sets have been implemented using similar gesture techniques, for example, T-CUBE (Venolia and Neilberg 1994) and Graffiti (www.palm.com). Beyond text input, non-linear gesture input has been used for a wide range of application areas including air traffic control (Chatty and Lecoanet 1996) . The GRANDMA toolkit allows gesture recognition to be added to interfaces by having the system developer provide examples of gestures and their associated interface actions (Rubine 1995). Despite the wide range of gesture systems developed, there has been relatively little work on characterising the physical properties of gestures used by recognition software. Dulberg, Amant and Zettlemoyer (1999) compared simple linear flick gestures with normal button clicks and keyboard shortcuts. In tasks that involved flicking towards abstract targets, they showed gestures to be 26% faster than button selection, but not reliably faster than key-bindings. Users also found the gestures easy to learn and accurate with only 4 errors from 3300 trials. In their six-subject informal study of flick gestures for window-management tasks (redirecting keyboard focus to items on the Microsoft Windows desktop) subjects reported no problems with learning the technique, and five of the six participants said they would use it if available.
doi:10.1080/01449290512331321866 fatcat:liun46yxpfgylcbgt2ryt2l4mi