Automated calibration of somatosensory stimulation using reinforcement learning

• Published in: Journal of neuroengineering and rehabilitation Vol. 20; no. 1; pp. 1 - 16
• Main Authors: Borda, Luigi, Gozzi, Noemi, Preatoni, Greta, Valle, Giacomo, Raspopovic, Stanisa
• Format: Journal Article
• Language: English
• Published: London BioMed Central 26.09.2023; BMC
• Subjects: Algorithms; Amputation; Artificial intelligence; Automatic calibration; Automation; Bioengineering; Calibration; Closed loop systems; Electrical stimulation; Electrical stimuli; Electrodes; Feedback; Graphical user interface; Machine learning; Mapping; Neural prostheses; Neuromodulation; Optimization techniques; Parameter identification; Peripheral nerves; Peripheral neuropathy; Prostheses; Prosthetics; Reinforcement; Reinforcement learning; Sensation; Sensory feedback; Stimulation; TENS
• ISSN: 1743-0003; 1743-0003
• DOI: 10.1186/s12984-023-01246-0

BackgroundThe identification of the electrical stimulation parameters for neuromodulation is a subject-specific and time-consuming procedure that presently mostly relies on the expertise of the user (e.g., clinician, experimenter, bioengineer). Since the parameters of stimulation change over time (due to displacement of electrodes, skin status, etc.), patients undergo recurrent, long calibration sessions, along with visits to the clinics, which are inefficient and expensive. To address this issue, we developed an automatized calibration system based on reinforcement learning (RL) allowing for accurate and efficient identification of the peripheral nerve stimulation parameters for somatosensory neuroprostheses.MethodsWe developed an RL algorithm to automatically select neurostimulation parameters for restoring sensory feedback with transcutaneous electrical nerve stimulation (TENS). First, the algorithm was trained offline on a dataset comprising 49 subjects. Then, the neurostimulation was then integrated with a graphical user interface (GUI) to create an intuitive AI-based mapping platform enabling the user to autonomously perform the sensation characterization procedure. We assessed the algorithm against the performance of both experienced and naïve and of a brute force algorithm (BFA), on 15 nerves from five subjects. Then, we validated the AI-based platform on six neuropathic nerves affected by distal sensory loss.ResultsOur automatized approach demonstrated the ability to find the optimal values of neurostimulation achieving reliable and comfortable elicited sensations. When compared to alternatives, RL outperformed the naïve and BFA, significantly decreasing the time for mapping and the number of delivered stimulation trains, while improving the overall quality. Furthermore, the RL algorithm showed performance comparable to trained experimenters. Finally, we exploited it successfully for eliciting sensory feedback in neuropathic patients.ConclusionsOur findings demonstrated that the AI-based platform based on a RL algorithm can automatically and efficiently calibrate parameters for somatosensory nerve stimulation. This holds promise to avoid experts’ employment in similar scenarios, thanks to the merging between AI and neurotech. Our RL algorithm has the potential to be used in other neuromodulation fields requiring a mapping process of the stimulation parameters.Trial registration: ClinicalTrial.gov (Identifier: NCT04217005)