Hand Sign Recognition System Based on EIT Imaging and Robust CNN Classification

• Published in: IEEE sensors journal Vol. 22; no. 2; pp. 1729 - 1737
• Main Authors: Ben Atitallah, Bilel, Hu, Zheng, Bouchaala, Dhouha, Hussain, Mohammed Abrar, Ismail, Amir, Derbel, Nabil, Kanoun, Olfa
• Format: Journal Article
• Language: English
• Published: New York IEEE 15.01.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; Classification; Conductivity; convolutional neural network; Current measurement; EIT; Electrical impedance; Electrical impedance tomography; Electrodes; Forearm; hand sign detection; Image classification; Image reconstruction; Impedance; Muscles; Muscular function; Recognition; regularized Gauss-Newton algorithm; Robustness; sign language; Signs; softmax; support vector machine; Support vector machines; Three dimensional models; virtual hand control; Visualization; Water tanks
• ISSN: 1530-437X; 1558-1748
• DOI: 10.1109/JSEN.2021.3130982

Hand sign recognition is gaining importance in human-human and in human-machine communication and interaction. Electrical Impedance Tomography (EIT) is thereby very interesting as it provides information on impedance changes in depth of the Section of the arm, which infers muscle contractions. This paper introduces an EIT imaging system for hand sign recognition and monitoring having a low complexity and including an electronic interface with 8 electrodes placed on the forearm, a Gauss-Newton image reconstruction algorithm, a robust CNN based hand sign classification and a virtual hand model for visualization. A database has been collected for EIT measurements in pole mode taken by eight subjects performing the American sign language numbers from 0 to 9. The overall imaging system is validated using a water tank system, where conductive objects can be changed in properties and positions. The correspondence between the reconstructed images and the expected muscle behavior for the hand signs is investigated. A robust Convolutional Neural Network (CNN) classification algorithm was implemented and optimized by implementing an Adam optimizer and conducting a dedicated study to avoid overfitting. The results obtained by CNN are compared to the results by a Support Vector Machine (SVM), and a Softmax classifier. They show a classification accuracy of 95.94%, 75.61%, and 62.9% respectively. In term of subject dependency, the system using the CNN model shows a higher performance, as the accuracy decreases only by 0.72% while increasing the number of subjects from one to eight. Finally, for visualization, a 3D virtual hand model is designed and controlled based on detected hand signs.