A system for understanding the content of street signs using finger-tracking

Papoulakis, G 2011, A system for understanding the content of street signs using finger-tracking , PhD thesis, Salford : University of Salford.

[img] PDF
Restricted to Repository staff only until 31 July 2023.

Download (22MB) | Request a copy


The work describes the development of a computer vision system which detects and interprets the text on signs in a scene. A key component in this system is the use of finger detection to select a particular sign in the scene, or text on a sign. The system is comprised of a Head-Mounted Display equipped with a light-weight camera which are connected to a wearable computer or a laptop. A set of Image Processing, Pattern Recognition and Optical Character Recognition methods are aggregated to process the sign's writing in real time. The system employs symbol recognition to detect graphics on a sign and OCR to process text. These merge to annotate the sign on the HMD for the user to read. Implementation details of the system are presented along with results gathered through experimental usability tests with prototype applications. A method to detect a finger-pointing hand gesture is used which together with a finger-tracking method serve as means of making a selection of the content on a sign. The gesture is used in two ways, one being to simply point at a sign's content with the finger, while the other method involves flexing the finger to perform an action resembling a mouse click. The two selection methods are compared through a series of indoor usability experiments where small teams of 5-10 users try both methods with single and multiple attempts on mock information signs. Statistical comparisons of subjective and objective data indicate the conditions that affect the time performance, comfort and preference of each method when selecting the entire writing of a sign or part of it. The method of pointing to select the entire writing on a sign showed constant preference and achieved higher performance. This was tested outdoors with real information signs on the campus of the University of Salford to confirm the comparison's results.

Item Type: Thesis (PhD)
Contributors: Ritchings, T (Supervisor)
Schools: Schools > School of Computing, Science and Engineering
Depositing User: Institutional Repository
Date Deposited: 03 Oct 2012 13:34
Last Modified: 04 Aug 2022 11:24
URI: http://usir.salford.ac.uk/id/eprint/26853

Actions (login required)

Edit record (repository staff only) Edit record (repository staff only)