Using Probabilistic Corrections to Account for Abstractor Agreement in Medical Record Reviews

• Published in: American journal of epidemiology Vol. 165; no. 12; pp. 1454 - 1461
• Main Authors: Lash, Timothy L., Fox, Matthew P., Thwin, Soe Soe, Geiger, Ann M., Buist, Diana S. M., Wei, Feifei, Field, Terry S., Yood, Marianne Ulcickas, Frost, Floyd J., Quinn, Virginia P., Prout, Marianne N., Silliman, Rebecca A.
• Format: Journal Article
• Language: English
• Published: Cary, NC Oxford University Press 15.06.2007; Oxford Publishing Limited (England)
• Subjects: Abstracting and Indexing as Topic; Aged; Aged, 80 and over; Analysis. Health state; Biological and medical sciences; Biostatistics; Breast cancer; breast neoplasms; Breast Neoplasms - epidemiology; Breast Neoplasms - pathology; Breast Neoplasms - surgery; Cancer; Comorbidity; Data collection; Data Interpretation, Statistical; epidemiologic methods; Epidemiology; Female; General aspects; Gynecology. Andrology. Obstetrics; Humans; Mammary gland diseases; Mastectomy - methods; Medical Records; Medical sciences; Miscellaneous; Neoplasm Recurrence, Local - epidemiology; Probability; Public health. Hygiene; Public health. Hygiene-occupational medicine; Tumors; United States - epidemiology; United States; epidemiologic methods; medical records; data collection; breast neoplasms; Human; Methodology; Breast tumor; Breast cancer; Malignant tumor; Review; Method; Epidemiology; Mammary gland diseases; Medical record; Corrections; Bibliographic review; Public health; Data gathering
• ISSN: 0002-9262; 1476-6256
• DOI: 10.1093/aje/kwm034

The quality of medical record abstracts is often characterized in a reliability substudy. These results usually indicate agreement, but not the extent to which lack of agreement affects associations observed in the complete data. In this study, medical records were reviewed and abstracted for patients diagnosed with stage I or stage II breast cancer between 1990 and 1994 at one of six US Cancer Research Network sites. For a subsample, interrater reliability data were available. The authors calculated conventional hazard ratios and 95% confidence intervals for the association of demographic, tumor, and treatment characteristics with recurrence rate. These conventional estimates of effect were compared with three sets of estimates and 95% simulation intervals that took account of the uncertainty assessed by lack of agreement in the reliability substudy. The rate of recurrence was associated with increasing cancer stage and with treatment modality but not with demographic characteristics. The hazard ratios and simulation intervals that took account of the reliability data showed that the simulation interval grew wider as the sources of uncertainty taken into account grew more complete, but the associations expected a priori remained readily apparent. While many investigators use reliability data only as a metric for data quality, a more thorough approach can also quantitatively depict the uncertainty in the observed associations.