Neural networks for automatic scoring of arthritis disease activity on ultrasound images

• Published in: Rheumatic & musculoskeletal diseases open Vol. 5; no. 1; p. e000891
• Main Authors: Andersen, Jakob Kristian Holm, Pedersen, Jannik Skyttegaard, Laursen, Martin Sundahl, Holtz, Kathrine, Grauslund, Jakob, Savarimuthu, Thiusius Rajeeth, Just, Søren Andreas
• Format: Journal Article
• Language: English
• Published: England EULAR 01.03.2019; BMJ Publishing Group LTD; BMJ Publishing Group
• Series: Original article
• Subjects: Algorithms; Arthritis - diagnosis; Arthritis, Rheumatoid - diagnosis; Artificial Intelligence; automatic scoring; Biopsy; Case-Control Studies; Classification; Clinical medicine; Clinical trials; convolutional neural networks; Deep Learning; disease activity; Humans; Imaging; Neural networks; Neural Networks, Computer; OMERACT-EULAR Synovitis Scoring system; Rheumatoid arthritis; Rheumatology; Severity of Illness Index; Synovitis - diagnosis; Ultrasonic imaging; Ultrasonography - methods; Ultrasonography, Doppler; ultrasound; United States > US; deep learning; automatic scoring; disease activity; convolutional neural networks; OMERACT-EULAR Synovitis Scoring system; rheumatoid arthritis; artificial intelligence; ultrasound
• ISSN: 2056-5933; 2056-5933
• DOI: 10.1136/rmdopen-2018-000891

BackgroundThe development of standardised methods for ultrasound (US) scanning and evaluation of synovitis activity by the OMERACT-EULAR Synovitis Scoring (OESS) system is a major step forward in the use of US in the diagnosis and monitoring of patients with inflammatory arthritis. The variation in interpretation of disease activity on US images can affect diagnosis, treatment and outcomes in clinical trials. We, therefore, set out to investigate if we could utilise neural network architecture for the interpretation of disease activity on Doppler US images, using the OESS scoring system.MethodsTwo state-of-the-art neural networks were used to extract information from 1342 Doppler US images from patients with rheumatoid arthritis (RA). One neural network divided images as either healthy (Doppler OESS score 0 or 1) or diseased (Doppler OESS score 2 or 3). The other to score images across all four of the OESS systems Doppler US scores (0–3). The neural networks were hereafter tested on a new set of RA Doppler US images (n=176). Agreement between rheumatologist’s scores and network scores was measured with the kappa statistic.ResultsFor the neural network assessing healthy/diseased score, the highest accuracies compared with an expert rheumatologist were 86.4% and 86.9% with a sensitivity of 0.864 and 0.875 and specificity of 0.864 and 0.864, respectively. The other neural network developed to four class Doppler OESS scoring achieved an average per class accuracy of 75.0% and a quadratically weighted kappa score of 0.84.ConclusionThis study is the first to show that neural network technology can be used in the scoring of disease activity on Doppler US images according to the OESS system.