Video-based feature extraction techniques for isolated Arabic sign language recognition

This paper presents various spatio-temporal feature extraction techniques with applications to recognition of isolated Arabic Sign Language (ArSL) gestures. The temporal features of a video-based gesture are extracted through forward image predictions. The prediction errors are thresholded and accumulated into one image that represents the sequence motion. The motion representation is then followed by spatial domain feature extractions, namely; 2-D DCT followed by Zonal coding or Radon transformation followed by ideal low pass filtering of the projected spatial features. The proposed feature extraction scheme was complemented by simple classification techniques, namely, KNN and Bayesian classifiers. Experimental results showed superior classification performance ranging from 97% to 100% recognition rates. To validate our proposed technique, we conducted a series of experiments using the classical way of classifying data with temporal dependencies. Namely, Hidden Markov Models (HMMs). Here, the features are the consecutive binarized image differences, each of which is followed by spatial domain feature extraction schemes. Experimental results revealed that the proposed feature extraction scheme combined with simple KNN or Bayesian classification yields comparable results to the classical HMM-based scheme.