Object stiffness recognition and vibratory feedback without ad-hoc sensing on the Hannes prosthesis: A machine learning approach

• Published in: Frontiers in neuroscience Vol. 17; p. 1078846
• Main Authors: Bruni, Giulia, Marinelli, Andrea, Bucchieri, Anna, Boccardo, Nicolò, Caserta, Giulia, Di Domenico, Dario, Barresi, Giacinto, Florio, Astrid, Canepa, Michele, Tessari, Federico, Laffranchi, Matteo, De Michieli, Lorenzo
• Format: Journal Article
• Language: English
• Published: Switzerland Frontiers Research Foundation 16.02.2023; Frontiers; Frontiers Media S.A
• Subjects: Activities of daily living; Amputation; closed-loop control; Engineering Sciences; Feedback; Hannes prosthetic hand; Machine learning; Neuroscience; non-linear logistic regression; Phantom limb; Proprioception; Prostheses; Prosthetics; Recovery of function; Sensors; Skin; stiffness recognition; vibromotor; vibrotactile feedback; Hannes prosthetic hand; vibromotor; vibrotactile feedback; closed-loop control; non-linear logistic regression; stiffness recognition
• ISSN: 1662-453X; 1662-4548; 1662-453X
• DOI: 10.3389/fnins.2023.1078846

In recent years, hand prostheses achieved relevant improvements in term of both motor and functional recovery. However, the rate of devices abandonment, also due to their poor embodiment, is still high. The embodiment defines the integration of an external object - in this case a prosthetic device - into the body scheme of an individual. One of the limiting factors causing lack of embodiment is the absence of a direct interaction between user and environment. Many studies focused on the extraction of tactile information custom electronic skin technologies coupled with dedicated haptic feedback, though increasing the complexity of the prosthetic system. Contrary wise, this paper stems from the authors' preliminary works on multi-body prosthetic hand modeling and the identification of possible intrinsic information to assess object stiffness during interaction. Based on these initial findings, this work presents the design, implementation and clinical validation of a novel real-time stiffness detection strategy, without sensing, based on a Non-linear Logistic Regression (NLR) classifier. This exploits the minimum grasp information available from an under-sensorized and under-actuated myoelectric prosthetic hand, Hannes. The NLR algorithm takes as input motor-side current, encoder position, and reference position of the hand and provides as output a classification of the grasped object (no-object, rigid object, and soft object). This information is then transmitted to the user vibratory feedback to close the loop between user control and prosthesis interaction. This implementation was validated through a user study conducted both on able bodied subjects and amputees. The classifier achieved excellent performance in terms of F1Score (94.93%). Further, the able-bodied subjects and amputees were able to successfully detect the objects' stiffness with a F1Score of 94.08% and 86.41%, respectively, by using our proposed feedback strategy. This strategy allowed amputees to quickly recognize the objects' stiffness (response time of 2.82 s), indicating high intuitiveness, and it was overall appreciated as demonstrated by the questionnaire. Furthermore, an embodiment improvement was also obtained as highlighted by the proprioceptive drift toward the prosthesis (0.7 cm).