Yantha for her contribution to implementation of the math and social studies activities and the I Can Centre for Assistive Technology team for their advice and assistance. Abstract Purpose A 12 year old girl, Emily, with complex communication needs and severe physical limitations, controlled a Lego robot from a speech generating device (SGD) to do various 'handson' academic activities. Emily's teacher and assistive technology (AT) team thought that controlling a robot would motivate Emily to

more »

... e her SGD more'. Method A descriptive case study was used because the integration of communication and manipulation technologies is not yet understood. Target activities and goals were chosen by Emily's teacher and AT team. Emily performed several manipulative math activities and engaged in an 'acting' activity aimed at increasing her message length. The competency skills needed to control a robot from the SGD were examined, as well as stakeholder satisfaction with the robot system. Results Emily generated up to 0.4 communication events and 7 robot commands per minute in the activities. Her length of utterance was usually one-word long, but she generated two-and threeword utterances during some activities. Observations of Emily informed a framework to describe the competency skills needed to use SGDs to control robots. Emily and her teacher expressed satisfaction with robot use. Conclusion Robot use could motivate students to build SGD operational skills and learn educational concepts. Use of robots in 'hands-on' academic activities: A case study examining the skills required to use a speech generating device to control a robot 1.0 Background Current teaching practices call for students to have inquiry-based learning opportunities through hands-on activities and to communicate what they have learned [1] [2] [3] . This hands-on, enactive approach has many benefits: it increases student motivation and engagement (e.g. [4] ), improves student understanding while they use concrete examples and 'verbalize to internalize ' [5, p 145]; and teachers can ascertain student's level of understanding as students ask for help or talk aloud [6] . Children with complex communication needs (CCN) who also experience severe physical limitations may find it difficult to actively engage in hands-on activities, and might be excluded or take a passive role [7, 8] . Physical limitations can affect the motor control required for accurate pointing, grasping, and manipulating physical objects [9] and children's only choice may be to watch others doing the manipulation for them. Children with CCN may use augmentative and alternative communication (AAC) methods such as speech generating devices (SGDs) to direct the manipulation of objects by telling a classmate which objects to move [as in math in 7], or, they may use gestures such as pointing, head nods and shakes, or eye gaze to respond to options presented to them by a classmate or teacher. However, if children with disabilities have opportunities to themselves perform hands-on activities and communicate they are likely be more engaged in the activity, and the enriched experience may augment the learning realised. Assistive robot systems can help children with physical limitations to manipulate objects in school-based activities. Movement commands can be stored and replayed, allowing the robot to repetitively perform actions to complete a task; new movement commands can be programmed as 4 children move on to new topics. For example, assistive robots have been developed for: science lab activities such as bringing items closer for sensory inspection [10, 11] , putting a glass over a burning candle to extinguish it [12] or mixing solutions, planting seeds, and plugging in electrical wires to make a radio [13] ; math activities such as drawing lines to match questions and answers on a worksheet [14] ; and art activities such as pasting items onto an art collage [13] . In most of the studies cited here, the investigator examined the feasibility of using a robot to perform a task. The number of participants in the studies ranged from one to seven, and their ages ranged from seven to 29 years. Participants were described as having moderate to severe orthopeadic disabilities, arthrogryposis, muscular distrophy, and cerebral palsy. Interfaces with which children sent commands to control the robot included: a five-slot switch [10], three push buttons [12], two joysticks [13], and single switch scanning [14]. Most studies were qualitative observational trials; in cases where data were collected they represented: number of interactions, accuracy, response time [10], time to perform tasks, and number of external operator interventions required [13]. However, Howell, Martz, and Stanger [11] measured students' performance with respect to curriculum concepts; pre-and posttests were given on the five senses, but students performed so well on the pretest that there was no room to improve. The high cost of robotic systems (e.g. $12,000 to $30,000) make them unaffordable for most people, so these studies are not easily replicated. More recent studies in the area of play have used inexpensive infrared controlled Lego Mindstorms robots. For example, children with cerebral palsy used Lego RCX robots in activities like knocking over blocks, bringing princesses to a ball, and delivering valentines to friends [15] . The children made the robot move by pressing one to four switches using various body parts, like the head, for example. Ljunglof et al. [16] developed a robot system using a Bluetooth controlled Lego NXT robot disguised as a bumble bee to draw 5 shapes on the floor. Children with autism chose commands through a touch screen on a tablet computer, and the robot asked the children for more information before implementing a command. None of these robot activities involved AAC during robotic manipulation tasks; although the robot communicated in the Ljunglof et al. [18] study, the child did not choose a response using AAC. The following observations in robotic play studies illustrate the importance of providing AAC during play activities. Children generally increased the number of vocalizations during and after robotic play interventions, possibly indicating a desire to comment [17] . In [15] the participant used an SGD, but it had to be removed so that she could use switches to control the robot. This led to the missed communication opportunities described in [18] . For example, on one occasion the participant appeared to be randomly controlling the robot and not following the prescribed play routine. When her mother interpreted the child's nonverbal communication for the investigators, it became evident that her behaviour was intentional and she was being innovative in her robot play. Had an SGD been available during the robot activity, she would have been able to communicate her intention. Disengaging from manipulation activities in order to communicate or disengaging from communication in order to manipulate is undesirable, but Light and Drager [19] highlighted research that is beginning to address this issue in play activities. For example, for children who have no physical limitations, communication symbols have been integrated into the play area so children do not have to coordinate their attention between manipulating the play items and commenting via a separate AAC system [19] . For children who have more severe physical disabilities, a system is needed which provides both communication and manipulation from the same access interface (e.g., switches, eye tracking). This is possible through the infrared or BlueTooth output feature built into many SGDs. used her SGD. Context-dependent communicators can reliably use words, signs, or pictures to represent a concept or meaning, but they are limited to particular contexts or partners [23] . Emily rarely initiated communication with the SGD but she would readily engage in conversation with nonverbal communication such as smiles, laughter, and head nods and shakes. In social conversation with familiar partners such as her AT team, her teachers, and her education assistant (EA), Emily typically generated utterances requiring only one selection on the SGD, for example, stored social comments such as 'HOW ARE YOU?' or responses one word in length such as 'mom'. When motivated, she sometimes composed two or three symbol messages (e.g. 'I go swimming'). The notation throughout the paper will represent stored phrases in capital letters and words, phrases, or sentences composed by Emily from her core vocabulary in lower case letters. Emily was unfamiliar with the Unity vocabulary on her device, and when requested to compose a sentence, she would spend several minutes searching for words and required cueing from her communication partner regarding the symbol pathways she must follow to locate them. Emily was in an integrated grade six classroom (with students aged 11 to 13 years), but did not study the grade six curriculum. No formal testing was performed as part of this study, but Emily's teacher reported that psychological testing prior to the study indicated that Emily had mild to moderate intellectual impairment. The teacher's opinion was that the impairment was due to reduced opportunities for learning. An educational assistant (EA) provided academic and personal assistance to Emily and one other student. Emily performed individualized reading, writing, and math activities with her EA and group activities such as acting in drama class with her peers. Prior to receiving an SGD, Emily had no means for written communication, hence her skills were delayed. She wrote short sentences with assistance from her EA who provided sentence frames for copying. Her teacher reported that Emily understood counting numbers up to five, and was requiring her to make two or three words sentences. Students in Emily's social studies class were creating PowerPoint v presentations about ancient Greece, so Emily was allowed to act out a Greek myth in an alternate presentation format. 'Theseus and the Minotaur' was chosen since it had potential robot action such as manoeuvring through a labyrinth [25] . Emily was expected to write the story during social studies class using writing supports (e.g. modeling and sentence frames) with her EA and then act out the myth during the robot sessions. However, Emily's EA reported that Emily lost interest in composing the myth after only a few minutes of writing. In order to continue with the intended activity of acting out the myth, the story narration (the story background) was written by author 1 and uploaded into the notepad on Emily's SGD, thus allowing Emily to read aloud the narration as she stepped through each line on the notepad. In the sessions she acted out the movements with the car-like robot as 'Theseus' and the robot arm as the 'Minotaur' and she made up utterances for the characters (e.g., greetings) and spoke them using the SGD. Small toys were used as other characters and props. A classmate shared a picture of the Minotaur which was used as the costume on the robot arm. A movie was produced by author 1 by videotaping Emily's narration, robot movements, and spoken utterances, and then editing the videos with Windows Movie Maker vi . Time periods when Emily was scanning to the robot commands or vocabulary were omitted from the final movie, resulting in a finished product five minutes in length. Prompts regarding robot control from least to most support (e.g. 'try it', 'you want to turn left,' 'you use the blue left arrow to turn left towards the blue arm') were given to Emily by the RA as needed. To become familiar with the new prestored phrases on the SGD robot page, Emily selected each one prior to the activities to learn what it would say. In the activities, if Emily did 14 not respond to a natural cue, then the RA prompted Emily by asking what she thought she could say in the situation. If Emily indicated that she did not know what to say, the RA suggested examples from which to choose. Cueing for core vocabulary was given as needed by the RA by looking up symbol pathways on PASS vii demonstration software for the Vanguard on a laptop computer. After the study, Emily and her teacher were asked what they thought about using the robots in the activities. The interviews were unstructured and Emily responded using her SGD and the teacher responded via email. Data collection and analysis To examine the amount Emily used the SGD and the length of her messages, the communication and robot manipulation events in each activity were tracked, with her permission, using the SGDs built-in Language Activity Monitor viii (LAM). The LAM logs SGD output, listing every character, word, and prestored phrase spoken, and every robot infrared command sent by the SGD. Communication events were also tracked using Morae Usability Analysis software ix by importing session videos into Morae and then the RA coded them manually. This step served to confirm the communication entries in the log and determine if some were caused by misselections. Communications regarding choosing the marker pen colour was excluded from analysis since they were not part of all activities. The communication rate was calculated as all communication output (word, prestored phrase, and repetition) divided by session length. The robot manipulation rate was calculated as all robot infrared program and direct control commands divided by session length. The SGD communication rate was graphed per session, and also versus infrared commands rate. Visual analysis of the graphs was used to examine (1) how much communication output was made by the participant in each activity, and (2) how the amount of 15 robot manipulation performed in the activity related to the amount of participant communication. The type of utterance was also coded from the videos (e.g. a prestored phrase from the SGD robot page or SGD language set, a letter or number character, or an utterance of one or more words composed using the core vocabulary). Whether Emily's utterance was independent or cued from the RA was also noted. Emily's accuracy at controlling the robot was calculated from photographs of the board games after Emily completed each game. Using the ImageJ x program, the trace left by the felt tip marker was digitized. The AT team had proposed that if Emily stayed within the borders 75% of the time, then they would believe that Emily intended to follow the board game. Thus, accuracy was calculated as the length travelled by the robot inside the board game borders divided by the total pathway length. All videos were observed by author 1 to identify the skills Emily demonstrated while using the SGD to control the robot. As a starting point, interactions were coded using Light's communication competence domains (e.g. linguistic, operational, social, and strategic) [21] , and themes were extracted from the emerging codes. Finally, the poststudy interview responses were examined for indications of satisfaction. NVivo software was used to aid the analyses. Results SGD use Emily's target selection accuracy on her SGD was 50% in trial 1 and 100% in trial 2. Figure 3a shows the rate of SGD communication output per session and figure 3b shows the rate of SGD communication output versus rate of robot infrared commands output. Figure 3b was divided into quadrants with lines arbitrarily chosen visually. The horizontal line separates low/high communication rates and the vertical line separates low/high infrared output rates. The 16 activities in the upper-left quadrant had comparatively high communication output and low robot manipulation, those in the lower-left had low communication output and low robot manipulation, and those in the lower-right had low communication output and high robot manipulation. [Insert figure 3 about here] The count of types of utterances Emily made in the activities is shown in table 2. In the drawing and board game activities she generally used only the prestored phrases on the SGD robot page and other utterances that counted as one 'word' in length. For instance, she used the robot command 'STOP' in a communicative way in session 5, made a one-word comment about the activity in session 6, and said 'I'M SORRY' and 'oops' when she drove the robot over the RA's game piece in the board game in session 7. Emily used letters to shorten the length of time to compose utterances, for example, instead of spelling out the name of the player who should go first in session 7, she said the first letter of the name. In the puzzle activity, Emily began by requesting pieces by saying 'beetle' or 'worm', but since it required so many switch hits to find those words, the RA suggested that she use 'b' and 'w' instead. Though it was expected that Emily would generate two-word utterances to request a piece and help from the RA, she only said one, 'b off'. This was because she learned to use the strategy that 'on' was implied when the robot was near the pickup location, and the piece to take 'off' was implied when she stopped the robot at her chosen location on the puzzle. Thus, the puzzle activity resulted in many 'b' and 'w' utterances, and Emily gazing intently at the RA to perform the implied 'on' or 'off' function. In the dot-to-dot activity Emily generated one-word comments about what the drawings looked like ('sun' and 'octopus'), but was motivated to compose a longer sentence to request to do a second dot-to-dot drawing ('Let's again'). [Insert table 2 about here] of the other activities. Extending from Light's AAC social competency domain [21] , the robot operator must be able to interact with others using robot actions. Emily was able to use the SGD to control the robot and interact in many ways with the RA-drawing, playing a board game, making a puzzle, and acting out a story. Emily also demonstrated social skills because she had access to the robot. She demonstrated her sense of humor when she drew circles around every number on the spider's web dot-to-dot drawing after the RA joked that an accidental circle looked like a flattened bug. She independently ensured that her robot characters were face-to-face with other characters when she was about to speak to them in the myth, despite the fact that she never had the opportunity to independently orient her manual wheelchair for face-to-face conversation. Extending from Light's AAC strategic competency domain [21] , the robot operator must determine the correct time to switch between robot and communication modes and choose the most efficient methods to accomplish tasks. Emily did not spontaneously switch between communication and robot control modes. She tended to stay in the robot control mode until an utterance was expected from her. However, Emily did learn some strategies for the most efficient methods to accomplish tasks. For example, she learned that pressing and holding her switch when selecting the direct forward command (e.g. forward one step) resulted in continuous robot movement. She used the 'STOP' infrared command on the SGD robot page (which said 'stop' as auditory feedback) rather than going to the word in her core vocabulary. Emily also demonstrated strategic preferences during the activities. For example, she preferred to use the robot arm to roll the die rather than using the random number generator on the SGD. Conversely, she preferred to play X's and O's on the SGD rather than drawing out the grid and making the O's with the robot. Satisfaction 20 Emily was an enthusiastic partner in all robot activities and was always motivated to keep working toward the completion of an activity. When asked specifically what she thought of doing the activities with the robots, she responded with positive statements (e.g. 'awesome' or 'THIS IS FUN'). The teacher was also positive about Emily's involvement in the study. She noted that the robots allowed Emily to link with the curriculum and with other students using unique, motivating, and fun technology. The reported that Emily's attention was sustained for a long time and Emily had opportunities to make decisions, use her thinking skills, and improve her head switch control of her communication device. The teacher reported that Emily's classmates were enthusiastic about the movie when it was shown in social studies class; one classmate said 'I wish I did that with my robot' (referring to his robot programming in science class). Discussion Emily appeared motivated to accomplish math and social studies activities using an SGD to control a robot. She participated eagerly in the activities, and asked that some activities be repeated, e.g. 'Let's again' in the dot-to-dot drawing. The teacher noted that Emily's attention was sustained in sessions of 60 minutes, for instance, while she was moving the robot and making up appropriate utterances to act out the Greek myth. This contrasts with Emily's loss of interest after a few minutes of writing the myth with her EA. Increased participant engagement is consistent with previous robot studies [26] . Figure 3 shows how much Emily 'used her device' during the activities; her communication rate varied from 0 to 0.38 events per minute and her infrared output rate varied from 1.9 to 7 events per minute. The maximum infrared output rate is about 15 times higher than her communication rate; this is logical because it took more time to search for and compose communication utterances than to move the robot. The high number of switch hits required for 21 infrared output could have contributed to the improvement in Emily's head switch control identified by her teacher. It takes a great deal of practice using switches for scanning to develop expert skills [27] . This project suggests that controlling a robot could be a motivating activity to engage students in practice using their switches, potentially providing carry-over gains in access to AAC and computers. Some of the math and social studies activities required more communication or manipulation than others. Figure 3b shows the manipulation (robot control) versus communication output trade-off. Emily made fewer word selections in more action-focused activities requiring more robot manipulation commands (e.g. dot-to-dot drawings and board games in the lower right quadrant of figure 3b), whereas Emily made more word selections in less manipulation-focused activities (e.g. puzzle and myth in the upper left quadrant of figure 3b) . Higginbotham, Bisantz, et al. [28] found a similar task effect with SGD output being lower for tasks requiring more physical actions. They found that the communication rate was highest for a narrative task, lower for an instruction-giving task involving drawing on a map, and lowest for a cooperative puzzle building task. If robots are used to motivate students to build SGD skills, the manipulative task requirements should be taken into account so that the student can focus on the intended skill to be practiced: When the goal is operational (e.g. improved switch use) the activity could be following pathways or making dot-to-dot drawings. When the goal is linguistic (e.g. increasing the length of utterance in words) the activity could be acting out a story with the robot taking on roles. Emily generated some utterances two or more words in length in some of the activities, e.g., 'Let's again' in the connect-the-dots drawing, in the word-based board game and the myth. Though she usually needed cueing to generate them, she was actively involved in the process. We hoped that Emily would independently achieve four-word utterances in these activities, as she