TY - GEN
T1 - Contactless Sensing for Recognizing Common Signs in ASL and BSL
AU - Fatima, Aisha
AU - Hameed, Hira
AU - Imran, Muhammad Ali
AU - Abbasi, Qammer H.
AU - Abbas, Hasan
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - Deaf-mute people use sign language to communicate using hand movements, body postures and facial emotions. Recognising automatic sign language is still a challenge and is in its early stage of research. For this, sensor-based and vision-based approaches are used; however, they have some limitations (privacy concerns, maintenance and ambient lighting). So, contactless sensing emerged as a promising solution to recognise automatic sign language. This work used contactless sensing to recognise ten common signs(I/me, You, Meet, Hello, Understand, Help, Sorry, Time, Candies/Sweets, and Thank you) used in american sign language and british sign language to make the system more efficient. To the best of our knowledge, recognising two sign languages is a novel work which is not done before. Raw data of a single participant is gathered, which is then converted into a spectrogram using signal processing. Afterward, six pre-trained deep learning models (EfficientNet, InceptionV3, MobileNet, ResNet5O, VGG16, and VGG19) are used for classification purposes. 92% accuracy of classification is achieved using ResNet5O and VGG19.
AB - Deaf-mute people use sign language to communicate using hand movements, body postures and facial emotions. Recognising automatic sign language is still a challenge and is in its early stage of research. For this, sensor-based and vision-based approaches are used; however, they have some limitations (privacy concerns, maintenance and ambient lighting). So, contactless sensing emerged as a promising solution to recognise automatic sign language. This work used contactless sensing to recognise ten common signs(I/me, You, Meet, Hello, Understand, Help, Sorry, Time, Candies/Sweets, and Thank you) used in american sign language and british sign language to make the system more efficient. To the best of our knowledge, recognising two sign languages is a novel work which is not done before. Raw data of a single participant is gathered, which is then converted into a spectrogram using signal processing. Afterward, six pre-trained deep learning models (EfficientNet, InceptionV3, MobileNet, ResNet5O, VGG16, and VGG19) are used for classification purposes. 92% accuracy of classification is achieved using ResNet5O and VGG19.
KW - American Sign Lan-guage
KW - British Sign Language
KW - Contactless Sensing
KW - Deep Learning
KW - Xethru X4M03 Radar
UR - https://www.scopus.com/pages/publications/85207060933
U2 - 10.1109/AP-S/INC-USNC-URSI52054.2024.10686773
DO - 10.1109/AP-S/INC-USNC-URSI52054.2024.10686773
M3 - Conference contribution
AN - SCOPUS:85207060933
T3 - IEEE Antennas and Propagation Society, AP-S International Symposium (Digest)
SP - 339
EP - 340
BT - 2024 IEEE International Symposium on Antennas and Propagation and INC/USNCURSI Radio Science Meeting, AP-S/INC-USNC-URSI 2024 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2024 IEEE International Symposium on Antennas and Propagation and INC/USNCURSI Radio Science Meeting, AP-S/INC-USNC-URSI 2024
Y2 - 14 July 2024 through 19 July 2024
ER -