A convolutional neural network to classify American Sign Language fingerspelling from depth and colour images

Ameen, SA and Vadera, S 2017, 'A convolutional neural network to classify American Sign Language fingerspelling from depth and colour images' , Expert Systems, 34 (3) , e12197.

[img] PDF - Accepted Version
Restricted to Repository staff only until 7 February 2018.

Download (917kB) | Request a copy
[img] Microsoft Word - Accepted Version
Restricted to Repository staff only

Download (1MB) | Request a copy

Abstract

Sign language is used by approximately 70 million1 people throughout the world, and an automatic tool for interpreting it could make a major impact on communication between those who use it and those who may not understand it. However, computer interpretation of sign language is very difficult given the variability in size, shape and position of the fingers or hands in an image. Hence, this paper explores the applicability of deep learning for interpreting sign language. The paper develops a convolutional neural network aimed at classifying fingerspelling images using both image intensity and depth data. The developed convolutional network is evaluated by applying it to the problem of finger spelling recognition for American Sign Language. The evaluation shows that the developed convolutional network performs better than previous studies and has precision of 82% and recall of 80%. Analysis of the confusion matrix from the evaluation reveals the underlying difficulties of classifying some particular signs which is discussed in the paper.

Item Type: Article
Schools: Schools > School of Computing, Science and Engineering > Salford Innovation Research Centre (SIRC)
Journal or Publication Title: Expert Systems
Publisher: Wiley
ISSN: 0266-4720
Related URLs:
Depositing User: S Vadera
Date Deposited: 26 Jan 2017 09:21
Last Modified: 09 Aug 2017 22:56
URI: http://usir.salford.ac.uk/id/eprint/41255

Actions (login required)

Edit record (repository staff only) Edit record (repository staff only)

Downloads

Downloads per month over past year