Image-based mobile service: automatic text extraction and translation

Jérôme Berclaz, Nina Bhatti, Steven J. Simske, John C. Schettino, Reiner Creutzburg, David Akopian
2010 Multimedia on Mobile Devices 2010  
We present a new mobile service for the translation of text from images taken by consumer-grade cell-phone cameras. Such capability represents a new paradigm for users where a simple image provides the basis for a service. The ubiquity and ease of use of cell-phone cameras enables acquisition and transmission of images anywhere and at any time a user wishes, delivering rapid and accurate translation over the phone's MMS and SMS facilities. Target text is extracted completely automatically,
more » ... ring no bounding box delineation or related user intervention. The service uses localization, binarization, text deskewing, and optical character recognition (OCR) in its analysis. Once the text is translated, an SMS message is sent to the user with the result. Further novelties include that no software installation is required on the handset, any service provider or camera phone can be used, and the entire service is implemented on the server side. For a definition of cloud computing, see As illustrated on Fig. 1 , the translation process is initiated by a user sending a picture by MMS (a). This MMS is received by the back-end server, the image extracted (b) and passed to the text extraction module. This latter module attempts to isolate text elements from the rest of the image. It produces a binary image (c) along with the coordinate information of
doi:10.1117/12.840279 fatcat:nko3t7gvrndbzja7o66r2cbqbi