Skip to main navigation menu Skip to main content Skip to site footer

Colour-based gesture recognition for American Sign Language via Hidden Markov Models


We present a new approach to gesture recognition for use in a sign
language learning environment. This method utilizes inexpensive
cloth gloves to alleviate the difficulty of hand detection and to allow
for feature creation. Salient colours identify the glove base and
fingertip markers, which are then used to extract a hand centroid
and a convex hull describing the fingertips for each hand. A Hidden
Markov Model is created for each sign, as well as an additional
threshold model created from all signs. When a candidate sign is
performed, the sign of the HMM that produces the greatest likelihood
is matched, provided it also exceeds the threshold model
likelihood. Isolated recognition testing of the training library indicated
76% accuracy, and continuous recognition testing showed
60% accuracy.