Statistical methods for comparing test positivity rates between countries: Which method should be used and why?

• Published in: Methods (San Diego, Calif.) Vol. 195; pp. 72 - 76
• Main Authors: Hittner, James B., Fasina, Folorunso O.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.11.2021
• Subjects: Bayes Theorem; Bayesian; Bolivia - epidemiology; COVID-19; COVID-19 - diagnosis; COVID-19 - epidemiology; COVID-19 Testing - statistics & numerical data; COVID-19 Testing - trends; Data Interpretation, Statistical; Frequentist; Humans; Republic of Korea - epidemiology; Research Design - statistics & numerical data; Research Design - trends; Statistical methods; Test positivity; TP rates; United States - epidemiology; Uruguay - epidemiology; Bolivia; United States; Uruguay; Republic of Korea; COVID-19; Statistical methods; TP rates; Test positivity; Frequentist; Bayesian
• ISSN: 1046-2023; 1095-9130
• DOI: 10.1016/j.ymeth.2021.03.010

•The Test Positivity (TP) rate is an important measure of COVID-19 illness burden.•Pairs of countries with similar versus discrepant TP rates were compared.•For discrepant TP rates, both frequentist and Bayesian methods indicated genuine between-country differences.•For similar TP rates (0.009 vs. 0.007), only the Bayesian method indicated no difference.•When TP rates are similar and sample sizes are large, frequentist methods can be misleading. The test positivity (TP) rate has emerged as an important metric for gauging the illness burden due to COVID-19. Given the importance of COVID-19 TP rates for understanding COVID-related morbidity, researchers and clinicians have become increasingly interested in comparing TP rates across countries. The statistical methods for performing such comparisons fall into two general categories: frequentist tests and Bayesian methods. Using data from Our World in Data (ourworldindata.org), we performed comparisons for two prototypical yet disparate pairs of countries: Bolivia versus the United States (large vs. small-to-moderate TP rates), and South Korea vs. Uruguay (two very small TP rates of similar magnitude). Three different statistical procedures were used: two frequentist tests (an asymptotic z-test and the ‘N-1’ chi-square test), and a Bayesian method for comparing two proportions (TP rates are proportions). Results indicated that for the case of large vs. small-to-moderate TP rates (Bolivia versus the United States), the frequentist and Bayesian approaches both indicated that the two rates were substantially different. When the TP rates were very small and of similar magnitude (values of 0.009 and 0.007 for South Korea and Uruguay, respectively), the frequentist tests indicated a highly significant contrast, despite the apparent trivial amount by which the two rates differ. The Bayesian method, in comparison, suggested that the TP rates were practically equivalent—a finding that seems more consistent with the observed data. When TP rates are highly similar in magnitude, frequentist tests can lead to erroneous interpretations. A Bayesian approach, on the other hand, can help ensure more accurate inferences and thereby avoid potential decision errors that could lead to costly public health and policy-related consequences.