Skip to the content

A system for understanding the content of street signs using finger-tracking

Papoulakis, G 2011, A system for understanding the content of street signs using finger-tracking , PhD thesis, Salford : University of Salford.

[img] PDF
Restricted to Repository staff only until 03 October 2014.

Download (21MB) | Request a copy

    Abstract

    The work describes the development of a computer vision system which detects and interprets the text on signs in a scene. A key component in this system is the use of finger detection to select a particular sign in the scene, or text on a sign. The system is comprised of a Head-Mounted Display equipped with a light-weight camera which are connected to a wearable computer or a laptop. A set of Image Processing, Pattern Recognition and Optical Character Recognition methods are aggregated to process the sign's writing in real time. The system employs symbol recognition to detect graphics on a sign and OCR to process text. These merge to annotate the sign on the HMD for the user to read. Implementation details of the system are presented along with results gathered through experimental usability tests with prototype applications. A method to detect a finger-pointing hand gesture is used which together with a finger-tracking method serve as means of making a selection of the content on a sign. The gesture is used in two ways, one being to simply point at a sign's content with the finger, while the other method involves flexing the finger to perform an action resembling a mouse click. The two selection methods are compared through a series of indoor usability experiments where small teams of 5-10 users try both methods with single and multiple attempts on mock information signs. Statistical comparisons of subjective and objective data indicate the conditions that affect the time performance, comfort and preference of each method when selecting the entire writing of a sign or part of it. The method of pointing to select the entire writing on a sign showed constant preference and achieved higher performance. This was tested outdoors with real information signs on the campus of the University of Salford to confirm the comparison's results.

    Item Type: Thesis (PhD)
    Contributors: Ritchings, T(Supervisor)
    Additional Information:
    Schools: Colleges and Schools > College of Science & Technology
    Colleges and Schools > College of Science & Technology > School of Computing, Science and Engineering
    Depositing User: Institutional Repository
    Date Deposited: 03 Oct 2012 14:34
    Last Modified: 18 Feb 2014 14:57
    URI: http://usir.salford.ac.uk/id/eprint/26853

    Document Downloads

    More statistics for this item...

    Actions (login required)

    Edit record (repository staff only)