Finger-Tapping Motion Recognition Based on Skin Surface Deformation Using Wrist-Mounted Piezoelectric Film Sensors

• Published in: IEEE Sensors Journal Vol. 24; no. 11; pp. 17876 - 17884
• Main Authors: Jomyo, Shumma, Furui, Akira, Matsumoto, Tatsuhiko, Tsunoda, Tomomi, Tsuji, Toshio
• Format: Journal Article
• Language: English; Japanese
• Published: New York IEEE 01.06.2024; Institute of Electrical and Electronics Engineers (IEEE); The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Biodegradable piezoelectric film sensors; biological signal processing; Deformation; Eyewear; human–machine interface (HMI); Input devices; Keyboards; Machine learning; Motion perception; motion recognition; Piezoelectric films; Piezoelectric transducers; Sensors; Signal measurement; Skin; Tendons; Watches; Wearable computers; wearable device; Wearable technology; Wrist
• ISSN: 1530-437X; 2379-9153; 1558-1748
• DOI: 10.1109/JSEN.2024.3386333

The miniaturization of computers has led to the development of wearable devices in the form of watches and eyeglasses. Consequently, the narrower screen size has raised the issue of operability for text input. This problem can be resolved using external input devices, such as physical keyboards. However, this can impair portability and accessibility. This study proposes a finger-tapping motion recognition system using wrist-mounted piezoelectric film sensors to realize an input interface with high wearability and not limited by screen size. In the proposed system, biodegradable piezoelectric film sensors, which are highly compatible with biological signal measurement, are attached to the palmar and dorsal surfaces of the wrist to measure minute skin surface deformation during tapping. The system detects the occurrence of tapping movements for each finger by preprocessing the measured signals and calculating the total activity of all channels. It also recognizes the type of finger movement based on machine learning. In the experiment, we measured ten different signals, including five-finger flexion and extension, for 11 subjects, to evaluate the effectiveness of the proposed method. According to the experimental results, tapping recognition accuracy for time-series data was 77.5%, assuming character input. In addition, the time difference between the detected and actual taps was approximately 50 ms on average. Therefore, the proposed method can be utilized as an input interface for wristband-type wearable devices.