Frame-wise detection of surgeon stress levels during laparoscopic training using kinematic data

• Published in: International journal for computer assisted radiology and surgery Vol. 17; no. 4; pp. 785 - 794
• Main Authors: Zheng, Yi, Leonard, Grey, Zeh, Herbert, Fey, Ann Majewicz
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.04.2022; Springer Nature B.V
• Subjects: Accuracy; Biomechanical Phenomena; Classification; Classifiers; Computer Imaging; Computer Science; Health Informatics; Humans; Imaging; Kinematics; Laparoscopy; Laparoscopy - education; Machine learning; Medicine; Medicine & Public Health; Neural Networks, Computer; Original Article; Pattern Recognition and Graphics; Radiology; Recurrent neural networks; Robotic surgery; Robotic Surgical Procedures; Stress; Surgeons; Surgery; Training; Vision; Deep learning; Surgical stress; Laparoscopic surgery; Motion analysis
• ISSN: 1861-6429; 1861-6410; 1861-6429
• DOI: 10.1007/s11548-022-02568-5

Purpose Excessive stress experienced by the surgeon can have a negative effect on the surgeon’s technical skills. The goal of this study is to evaluate and validate a deep learning framework for real-time detection of stressed surgical movements using kinematic data. Methods 30 medical students were recruited as the subjects to perform a modified peg transfer task and were randomized into two groups, a control group ( n =15) and a stressed group ( n =15) that completed the task under deteriorating, simulated stressful conditions. To classify stressed movements, we first developed an attention-based Long-Short-Term-Memory recurrent neural network (LSTM) trained to classify normal/stressed trials and obtain the contribution of each data frame to the stress level classification. Next, we extracted the important frames from each trial and used another LSTM network to implement the frame-wise classification of normal and stressed movements. Results The classification between normal and stressed trials using attention-based LSTM model reached an overall accuracy of 75.86% under Leave-One-User-Out (LOUO) cross-validation. The second LSTM classifier was able to distinguish between the typical normal and stressed movement with an accuracy of 74.96% with an 8-second observation under LOUO. Finally, the normal and stressed movements in stressed trials could be classified with the accuracy of 68.18% with a 16-second observation under LOUO. Conclusion In this study, we extracted the movements which are more likely to be affected by stress and validated the feasibility of using LSTM and kinematic data for frame-wise detection of stress level during laparoscopic training. The proposed classifier could be potentially be integrated with robot-assisted surgery platforms for stress management purposes