Nystagmus patterns classification framework based on deep learning and optical flow

• Published in: Computers in biology and medicine Vol. 153; p. 106473
• Main Authors: Kong, Sheng, Huang, Zheming, Deng, Weike, Zhan, Yinwei, Lv, Jujian, Cui, Yong
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.02.2023; Elsevier Limited
• Subjects: Benign paroxysmal positional vertigo; Benign Paroxysmal Positional Vertigo - complications; Blinking; Classification; Deep Learning; Eye movements; Eyelid; Feature extraction; Frames (data processing); Humans; Information processing; Infrared analysis; Iris segmentation; Machine learning; Model accuracy; Motion perception; Nystagmus; Nystagmus classification; Nystagmus, Pathologic - diagnosis; Occlusion; Optic Flow; Optical flow; Optical flow (image analysis); Pattern recognition; Semicircular Canals; Temporal variations; Vertigo; Vestibular system; Video; Deep learning; Optical flow; Iris segmentation; Nystagmus classification; Benign paroxysmal positional vertigo
• ISSN: 0010-4825; 1879-0534
• DOI: 10.1016/j.compbiomed.2022.106473

Benign paroxysmal positional vertigo (BPPV) is the most common vestibular peripheral vertigo disease characterized by brief recurrent vertigo with positional nystagmus. Clinically, it is common to recognize the patterns of nystagmus by analyzing infrared nystagmus videos of patients. However, the existing approaches cannot effectively recognize different patterns of nystagmus, especially the torsional nystagmus. To improve the performance of recognizing different nystagmus patterns, this paper contributes an automatic recognizing method of BPPV nystagmus patterns based on deep learning and optical flow to assist doctors in analyzing the types of BPPV. Firstly, we present an adaptive method for eliminating invalid frames that caused by eyelid occlusion or blinking in nystagmus videos and an adaptive method for segmenting the iris and pupil area from video frames quickly and efficiently. Then, we use a deep learning-based optical flow method to extract nystagmus information. Finally, we propose a nystagmus video classification network (NVCN) to categorize the patterns of nystagmus. We use ConvNeXt to extract eye movement features and then use LSTM to extract temporal features. Experiments conducted on the clinically collected datasets of infrared nystagmus videos show that the NVCN model achieves an accuracy of 94.91% and an F1 score of 93.70% on nystagmus patterns classification task as well as an accuracy of 97.75% and an F1 score of 97.48% on torsional nystagmus recognition task. The experimental results prove that the framework we propose can effectively recognize different patterns of nystagmus. •Extracting the motion information of the iris to detect the torsional movement of the eyeballs.•Extract the eye movement information by using the optical flow methods.•Use a deep learning-based methods to analyze nystagmus symptom.•Analyze the torsional nystagmus through optical flow methods.