In this paper, we propose a first-person localization and navigation system for helping blind and visually-impaired people navigate in indoor environments. The system consists of a mobile vision front end with a portable panoramic lens mounted on a smart phone, and a remote GPU-enabled server. Compact and effective omnidirectional video features are extracted and represented in the smart phone front end, and then transmitted to the server, where the features of an input image or a short video

more »

... ip are used to search a database of an indoor environment via image-based indexing to find both the location and the orientation of the current view. One-dimensional omnidirectional profiles, which capture rich vertical lines and additional features in both HSI and HSI gradient space, are used in the database modeling step for constructing the model of an indoor environment from its panoramic video sequences. In the navigation step, the same type of features of a short video clip, are used as key words for searching in the database in order to provide candidate of the possible locations of the user and then estimate the orientation of the current view. To deal with the high computational cost in searching a large database for a realistic navigation application, data parallelism and task parallelism properties are identified in the database indexing steps, and computation is accelerated by using multi-core CPUs and GPUs. Experiments on synthetic data and real data are carried out to demonstrate the capacity of the proposed system with respect to real-time response and robustness.