Architecture and Applications of Virtual Coaches

Daniel Siewiorek, Asim Smailagic, Anind Dey
2012 Proceedings of the IEEE  
The Evolution of Technology The confluence of several new technologies enabled a new generation of always attentive personalized systems called Virtual Coaches. A Virtual Coaches continuously monitors its users activities and surroundings, detects situations where intervention would be desirable, and offers prompt assistance. Presently available cognitive aids are simplistic, providing only scheduled reminders and rote instructions. Future virtual coaches will actually monitor user performance
more » ... f activities and provide appropriate feedback and encouragement. As the user's abilities change, the coach may reduce the number of verbal cues as the subject learns, or provide increased support as needed. A care provider could upload new capabilities to the virtual coach, as required, potentially without even an office visit. Virtual coaches also provide constant and consistent observation/monitoring, even on a clinician's guidance beyond episodic patient examinations. Virtual Coaches are the latest phase of a technology evolution over the past two decades. The advent of powerful microprocessors capable of running an operating system with real time responsiveness in small, energy efficient pages in the early 1990's enabled a new generation of personal computing systems that provided access to information any time, any where. Handheld Personal Digital Assistant (PDA) that could fit in a shirt pocket gave access to addresses, notes, and schedules via a new interface access modality featuring stylus and hand writing recognition (e.g. graffiti) and more recently touch screen and voice control (e.g. SIRI). Another novel technology, head mounted displays, enabled revolutionary new body worn systems, termed Wearable Computers [Siewiorek, Smailagic, Starner 2008], that were always on providing instantaneous access to reference information in application areas such as complex plant operations, manufacturing, maintenance, and group collaboration. MEMS (Mircro-electro-mechanical systems) created low cost, low energy sensors that could sense physical parameters such as acceleration, orientation, temperature, and light that, when coupled with signal processing and machine learning algorithms allowed personal systems to infer user context in Context Aware Systems. Section 2 provides background on one class of applications for virtual coaches -cognitive aids demonstrating the potential user population. Section 3 introduces the elements in virtual coach architecture with a summary of how the elements are configured in the example virtual coaches presented in later sections. The following two sections provide two detailed examples of virtual coaches that implement two basically different coaching models -rule based that derives the model through end user involvement in the design process and machine learning that derives a statistical model through labeled sensor data. 2 Section 6 provides three more coaching examples while Section 7 provides conclusions and future challenges. Desirable Attributes of Virtual Coaches as Cognitive Aids One important application domain for virtual coaches is in assisting individuals whose own cognitive capabilities have been impaired due to natural aging, illness or traumatic injuries. Recent estimates indicate that over 20 million Americans experience some form of cognitive impairment. This includes older Americans living alone (~4M), people with Alzheimer's (~4.5M), people with mild cognitive impairments (~6M older adults), survivors of stroke (~2.5M) and people with traumatic brain injury (~5.3M). Of the many challenges faced by older individuals, declines in memory and cognition are often most feared and have the largest negative impact on themselves and their family members Cognitive aids currently available are simplistic, providing only scheduled reminders and rote instructions. They operate open-loop without regard for the user's activities or environment. In contrast, Virtual Coaches monitor how the user performs activities, provides situational awareness and gives feedback and encouragement matched to their cognitive state and circumstances at the time. Consider the difference between a medication reminder that blindly sounds an alert everyday at noon versus a Virtual Coach that both realizes a user took their pill at 11:58 or in another situation, such as when they are having a conversation, and sets itself to vibrate mode. Other transformative features of a Virtual Coach include: • As the user learns, it reduces the number of and level of detail in the cues it provides; • It matches its level of support to the user as his abilities change; • A caregiver can upload new capabilities to the Virtual Coach, as required, without even an office visit; and, • It provides constant and consistent monitoring of adherence to a caregiver's instructions, enabling a deeper and more timely understanding of conditions beyond the episodic patient examinations available today. Virtual Coaches based on understanding of user situations and needs are also effective for applications aimed at larger populations. For example, cognitive support can assure safe use and compliance with instructions in rehabilitation and management of chronic illness. Many individuals are released from hospital to home with inadequate training for themselves or their family caregivers for the operation of newly prescribed home medical devices or following complex medical regimens. Failure to properly follow directions often results in expensive (to the insurer) re-hospitalizations. Similarly, understanding how to effectively motivate people toward healthy behaviors, such as proper diet and physical activity, can benefit broad segments of the general population. Virtual coaches can monitor for compliance, provide cognitive assistance, provide advice that is trusted and followed, and adapt to user capabilities that vary with time and circumstances. The next section discusses the architectural elements common to all virtual coaches. 3 Virtual Coach Architecture As indicated above, currently available aids are simplistic, providing only scheduled reminders and rote instructions. Virtual coaches monitor user performance of activities and user context (Sensor Processing), determines appropriate feedback (Coaching Model), and provides feedback and encouragement (User Engagement). A care provider could upload new capabilities to the virtual coach, as required (Prescription). Over time a customized personal interaction evolves (User Interaction). Figure 1 depicts how these five elements interact to create a Virtual Coach. Figure 1. Inter-relation of five elements of virtual coaches.
doi:10.1109/jproc.2012.2200560 fatcat:4whdpa2s4batlaxmopyct6kmiy