Observer agreement in the grading of oral epithelial dysplasia

• Published in: Community dentistry and oral epidemiology Vol. 31; no. 4; pp. 300 - 305
• Main Authors: D. J, Brothwell, D. W, Lewis, G, Bradley, I, Leong, R. C. K, Jordan, D, Mock, J. L, Leake
• Format: Journal Article
• Language: English
• Published: Oxford UK Munksgaard International Publishers 01.08.2003
• Subjects: Adult; Aged; Aged, 80 and over; Carcinoma in Situ - diagnosis; Carcinoma in Situ - pathology; Carcinoma, Squamous Cell - diagnosis; Carcinoma, Squamous Cell - pathology; Dentistry; Epithelium - pathology; Female; Humans; leukoplakia; Male; Middle Aged; Mouth Mucosa - pathology; Mouth Neoplasms - diagnosis; Mouth Neoplasms - pathology; Observer Variation; oral observer variation; oral pathology; Precancerous Conditions - diagnosis; Precancerous Conditions - pathology; Reproducibility of Results
• ISSN: 0301-5661; 1600-0528
• DOI: 10.1034/j.1600-0528.2003.00013.x

Objectives: To determine the extent of observer agreement in diagnosis of oral epithelial dysplasia (OED). Published studies of OED examiner agreement report relatively low agreement levels; however, these studies were limited by the methodologies employed. Methods: For this study, 64 slides were each independently examined twice by three oral pathologists. Consistency was assessed by determining intra‐ and interexaminer agreement. Conformity was assessed by using the modal diagnosis as a gold standard. Results: The group showed moderate interobserver agreement when grading the presence or absence of OED with a group‐simple kappa (Ks) of 0.51 (95% CI = 0.42–0.61), and substantial agreement when using a 5‐point ordinal scale with a group‐weighted kappa (Kw) of 0.74 (95% CI = 0.64–0.85). The group showed fair to substantial intraexaminer agreement when assessing the presence or absence of OED, with Ks ranging from 0.22 to 0.78, and showing almost a perfect agreement using a 5‐point ordinal scale, with Kw ranging from 0.82–0.96. Conformity with the comparison standard modal diagnosis was almost perfect, with pairwise Kw ranging from 0.81 to 0.92. Conclusion: Overall, there was substantial intra‐ and interobserver consistency and almost perfect conformity in the grading of OED. Appropriate statistical methods are necessary to determine the degree of observer agreement.