Touch and Deformation Perception of Soft Manipulators With Capacitive e-Skins and Deep Learning

• Published in: IEEE sensors journal Vol. 24; no. 21; pp. 36076 - 36084
• Main Authors: Hu, Delin, Dong, Huazhi, Liu, Zhe, Chen, Zhou, Giorgio-Serchi, Francesco, Yang, Yunjie
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.11.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Actuation; Actuators; Capacitance; Decoupling; Deep learning; Deformation; Deformation effects; deformation perception; Electrodes; Manipulators; Multilayer perceptrons; Recognition; Robot arms; Robot sensing systems; Robots; Sensors; Soft robotics; tactile sensing; Tactile sensors (robotics); Touch; Wires
• DOI: 10.1109/JSEN.2024.3458443
• ISSN: 1530-437X

Tactile sensing in soft robots remains particularly challenging because of the coupling between contact and deformation, which the sensor is subject to during actuation and interaction with the environment. This often results in severe mutual interference and makes disentangling tactile sensing and geometric deformation difficult. To address this problem, this article proposes a soft capacitive e-skin with a sparse electrode distribution and deep learning for information decoupling. Our approach successfully separates tactile sensing from geometric deformation, enabling touch recognition on a soft pneumatic actuator subject to both internal (actuation) and external (physical contact) forces. Using a multilayer perceptron (MLP), the proposed e-skin achieves 99.88% accuracy in touch recognition across a range of deformation and contact states. When complemented with prior knowledge, a transformer-based architecture effectively tracks the deformation of the soft actuator. The average distance (AD) error in positional reconstruction of the manipulator is as low as <inline-formula> <tex-math notation="LaTeX">2.905~\pm ~2.207 </tex-math></inline-formula> mm, even under operative conditions with different inflation states and physical contacts, which lead to additional signal variations and consequently interfere with deformation tracking. These findings represent a tangible way forward in developing AI-assistive e-skins that potentially can endow soft robots with proprioceptive and exteroceptive capabilities simultaneously.