Probabilistic spatio-temporal retrieval in smart spaces

Vivek Menon, Bharat Jayaraman, Venu Govindaraju
2013 Journal of Ambient Intelligence and Humanized Computing  
A 'smart space' is one that automatically identifies and tracks its occupants using unobtrusive biometric modalities such as face, gait, and voice in an unconstrained fashion. Information retrieval in a smart space is concerned with the location and movement of people over time. Towards this end, we abstract a smart space by a probabilistic state transition system in which each state records the probabilities of presence of individuals in various zones of the smart space. We carry out
more » ... d reasoning on the states in order to determine more accurately the occupants of the smart space. This leads to a data model based upon an occupancy relation in which time is treated discretely, owing to the discrete nature of events, but probability is treated as a real-valued attribute. Using this data model, we show how to formulate a number of spatio-temporal queries, focusing on the computation of probabilities, an aspect that is novel to this model. We present queries both in SQL syntax and also in CLP(R), a constraint logic programming language (with reals) which facilitates succinct formulation of recursive queries. We show that the answers to certain queries are better displayed in a graphical manner, especially the movement tracks of occupants of the smart space. We also define query-dependent precision and recall metrics in order to quantify how well the model is able to answer various spatio-temporal queries. We show that a query-Vivek Menon 2 V. Menon et al. dependent metric gives significantly better results for a class of occupancy-related queries compared with query-independent metrics. Keywords Smart Spaces · Abstract Framework · Biometrics · Recognition · Retrieval · Precision · Recall · Data Model · Spatio-temporal Queries · CLP(R) Introduction A smart space is a physical space embedded with intelligence and interfaced with humans in a natural way using vision, speech, gestures, and touch, rather than the traditional keyboard and mouse. The key to realizing this paradigm is identifying and tracking people in the space. The ability to identify and track people and answer questions about their whereabouts is critical to many applications. Such smart spaces are very important and beneficial in a number of settings, including homes for the elderly or disabled, office workplaces, and larger areas such as department stores, shopping complexes, train stations, and airports. In some spaces, most of the individuals are known or pre-registered (health-care monitoring) whereas in other spaces most of the individuals are unknown (homeland security). Let us consider two scenarios from real-life incidents: (1) An elderly resident in an assisted living facility wears an RFID badge to facilitate continuous monitoring of his presence. On one occasion, he enters the elevator alone but gets trapped due to a power failure. The RFID signals transmitted by his badge are not in the range of any receiver. Only much later, when the elevator resumes its service, is he discovered. (2) An intruder has managed to gain illegal entry into a secure facility which is monitored by surveillance cameras. After an intruder alert has been raised, the security personnel set out to find the intruder and relies on inputs from the control room personnel monitoring the facility through multiple video feeds. As the intruder no longer appears on any of the video feeds, the search team has no other option but to search each room. Automated approaches to transforming multimedia data from video surveillance feeds into a form suitable for information retrieval is a very challenging problem and spans multiple areas -video and audio processing, computer vision, spatio-temporal reasoning and data models. These scenarios also highlight the need for unobtrusive data gathering, where people go about their normal activities without being subject to a 'pause and declare' routine or the burden of RFID tags or badges. Identifying people from their face, gait and voice is more natural and less obtrusive and hence more suited in smart spaces. The overall goal of our research is to develop indoor smart spaces that can recognize and track their occupants as unobtrusively as possible and answer queries about their whereabouts. The sensors of interest in our work are video cameras and microphones that capture biometric modalities such as face, gait, and voice in an unconstrained fashion. In our previous research, we have focused on multimodal approaches to biometric recognition (Menon et al, 2010) as well as the integration of recognition and reasoning in order to develop a more robust approach to identification and tracking (Menon et al, 2011 (Menon et al, , 2012a . This paper extends our most recent work (Menon et al, 2012b) on spatio temporal querying in smart spaces and discusses the results of information retrieval and performance of a smart space from a querydependent perspective. While the basic data is about the location of individuals
doi:10.1007/s12652-013-0199-2 fatcat:rzfhmgswyzginbvczh26lscxiy