AB1575 A RELIABLE METHOD FOR ANNOTATING HAND AND WRIST X-RAYS FOR SUPERVISED MACHINE LEARNING ALGORITHMS

• Published in: Annals of the rheumatic diseases Vol. 82; no. Suppl 1; pp. 2020 - 2021
• Main Authors: Rambojun, A., Antony, A., Hughes-Roberts, Y., Tillett, W.
• Format: Journal Article
• Language: English
• Published: Kidlington Elsevier B.V 01.06.2023; Elsevier Limited
• Subjects: Algorithms; Annotations; Artificial intelligence; Bones; Computer vision; Discordance; Hand; Imaging; Learning algorithms; Machine learning; Metacarpal; Psoriatic arthritis; Radiography; Radius; Trapezium; Ulna; Wrist; X-rays; Artificial intelligence; Imaging; Psoriatic arthritis
• DOI: 10.1136/annrheumdis-2023-eular.3434
• ISSN: 0003-4967

Machine learning (ML) algorithms could facilitate the standardisation of joint damage assessment in Psoriatic Arthritis (PsA) and improve its accessibility in clinical and research settings. ML algorithms trained on manually annotated hand and wrist radiographs have promising performance characteristics[1]. A large volume of annotated radiographs is needed, and annotation is time consuming and subject to reliability issues given X-Rays (XRs) are 2D representations. To develop a reliable method for the annotation of hand and wrist bones on XRs in order to facilitate the development of supervised ML algorithms for joint damage detection. 10 bilateral hand and wrist XRs were selected at random from the Bath PsA XR database. 5 XRs were independently annotated by 3 annotators; (AA & WT (rheumatologist) and YHR (radiologist)) using the ASPAX software[2]. Annotations were visually inspected for areas of discordance and consensus annotation guidelines were developed. Annotation was repeated using the annotation guidelines on second set of 5 XRs. With annotator 1 (WT) representing ground truth, the mean error (ME; in pixels) of the annotation (deviation from ground truth) and the mean fractional error (MFE; corrects for the perimeter measurements of the bone), was estimated in pre- and post-training annotations. The ME and MFE within a single annotator (AA) were estimated in 5 radiographs after a 2-month interval. Visual inspection determined that the areas of discordance in annotation were the 1st interphalangeal joint, the metacarpal bases, the hamate and capitate bones, and the trapezium and trapezoid bones (Figure 1). The MFE between the annotators and ground truth improved for all bones following the development of annotation guidelines, with the largest improvement evident in the annotation of the metacarpal bones (Table 1). The intra-reader and inter-reader MFEs were comparable (Table 1). Standardised instructions may facilitate reliable hand and wrist bone annotation and enable the acquisition of large volumes of annotated training data for supervised ML algorithms. [1]Adwaye Rambojun, William Tillett, Tony Shardlow, Neill D. F. Campbell; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021, pp. 2043-2052 [2]Machine Learning and Rheumatic Diseases (Website) https://people.bath.ac.uk/amr62/Projects/malard/malard.html NIL. Adwaye Rambojun: None declared, Anna Antony Speakers bureau: Eli Lilly, AbbVie, Ynyr Hughes-Roberts: None declared, William Tillett Speakers bureau: Abbvie, Amgen, Eli Lilly, GSK, Janssen, Novartis, Pfizer, UCB, Consultant of: Abbvie, Amgen, Eli Lilly, GSK, Janssen, Novartis, Ono Pharma, Pfizer, UCB, Grant/research support from: Janssen, UCB, Pfizer, Eli-Lilly. [Display omitted] Table 1Reliability ExerciseExercise 1: Inter-rater errorExercise 2: Inter-rater errorExercise 2: Intra-rater errorMean Error (±sd)PixelsMean Fractional Error (±sd)Mean Error (±sd)PixelsMean Fractional Error (±sd)Mean Error (±sd)PixelsMean Fractional Error (±sd)Distal phalanges201.02 (119.282)3.97 x 10-4(3.747 x 10-3)550.54(85.201)1.66 x10-5(1.293 x10-5)1.39 x10-2(8.740 x10-3)2.40 x10-5(1.335 x10-5)Middle phalanges218.83 (119.891)7.43 x 10-4(3.232 x 10-3)544.37(76.012)4.33 x 10-4(1.178 x 10-3)7.85 x10-3(2.368 x10-3)1.02 x10-5(4.055 x10-5)Proximal phalanges231.48 (116.095)8.52 x10-6(8.113 x10-6)585.33(104.098)7.13 x10-6(3.119 x10-6)7.80 x10-3(3.554 x10-3)6.88 x10-6(3.972 x10-6)Metacarpals184.94(115.208)1.74 x 10-4(1.345 x 10-3)571.73(62.721)8.65 x10-6(4.308 x10-6)9.82 x10-3(6.102 x10-3)6.07 x10-6(3.307 x10-6)Radius232.90(122.620)1.75 x 10-4(2.086 x 10-4)552.12(79.655)3.25 x10-5(1.775 x10-5)1.83215 x10-2(9.617 x10-3)1.25 x10-5(6.534 x10-6)Ulna236.60(121.709)3.31 x 10-4(3.778 x 10-4)557.25(80.019)2.19 x10-5(1.793 x10-5)1.68 x10-2(1.842 x10-2)1.41 x10-5(1.656 x10-5)Carpal bonesa197.42 (141.548)2.75 x 10-4(3.009 x 10-4)Carpal bonesb542.31(77.896)8.89 x10-5(1.025 x 10-4)2.86 x10-2(3.285 x10-2)5.28 x10-5(7.673 x10-5)a. Trapezium, Trapezoid, Scaphoid, Lunate, Pisiform, Triquetrum, Hamate Capitateb. Trapezium, Hamate/Capitate, Lunate, Scaphoid