Correcting prevalence estimation for biased sampling with testing errors

Sampling for prevalence estimation of infection is subject to bias by both oversampling of symptomatic individuals and error‐prone tests. This results in naïve estimators of prevalence (ie, proportion of observed infected individuals in the sample) that can be very far from the true proportion of in...

Full description

Saved in:
Bibliographic Details
Published inStatistics in medicine Vol. 42; no. 26; pp. 4713 - 4737
Main Authors Zhou, Lili, Díaz‐Pachón, Daniel Andrés, Zhao, Chen, Rao, J. Sunil, Hössjer, Ola
Format Journal Article
LanguageEnglish
Published New York Wiley Subscription Services, Inc 20.11.2023
Subjects
active information
bias correction
COVID-19
maximum entropy
prevalence
sampling
sampling bias
testing errors
Online AccessGet full text

Cover

Abstract Sampling for prevalence estimation of infection is subject to bias by both oversampling of symptomatic individuals and error‐prone tests. This results in naïve estimators of prevalence (ie, proportion of observed infected individuals in the sample) that can be very far from the true proportion of infected. In this work, we present a method of prevalence estimation that reduces both the effect of bias due to testing errors and oversampling of symptomatic individuals, eliminating it altogether in some scenarios. Moreover, this procedure considers stratified errors in which tests have different error rate profiles for symptomatic and asymptomatic individuals. This results in easily implementable algorithms, for which code is provided, that produce better prevalence estimates than other methods (in terms of reducing and/or removing bias), as demonstrated by formal results, simulations, and on COVID‐19 data from the Israeli Ministry of Health.
AbstractList Sampling for prevalence estimation of infection is subject to bias by both oversampling of symptomatic individuals and error‐prone tests. This results in naïve estimators of prevalence (ie, proportion of observed infected individuals in the sample) that can be very far from the true proportion of infected. In this work, we present a method of prevalence estimation that reduces both the effect of bias due to testing errors and oversampling of symptomatic individuals, eliminating it altogether in some scenarios. Moreover, this procedure considers stratified errors in which tests have different error rate profiles for symptomatic and asymptomatic individuals. This results in easily implementable algorithms, for which code is provided, that produce better prevalence estimates than other methods (in terms of reducing and/or removing bias), as demonstrated by formal results, simulations, and on COVID‐19 data from the Israeli Ministry of Health.
Author Rao, J. Sunil
Zhao, Chen
Hössjer, Ola
Zhou, Lili
Díaz‐Pachón, Daniel Andrés
Author_xml – sequence: 1
  givenname: Lili
  surname: Zhou
  fullname: Zhou, Lili
  organization: Division of Biostatistics University of Miami Miami Florida USA
– sequence: 2
  givenname: Daniel Andrés
  orcidid: 0000-0001-6281-1720
  surname: Díaz‐Pachón
  fullname: Díaz‐Pachón, Daniel Andrés
  organization: Division of Biostatistics University of Miami Miami Florida USA
– sequence: 3
  givenname: Chen
  surname: Zhao
  fullname: Zhao, Chen
  organization: Division of Biostatistics University of Miami Miami Florida USA
– sequence: 4
  givenname: J. Sunil
  surname: Rao
  fullname: Rao, J. Sunil
  organization: Division of Biostatistics University of Minnesota Minneapolis Minnesota USA
– sequence: 5
  givenname: Ola
  surname: Hössjer
  fullname: Hössjer, Ola
  organization: Department of Mathematics Stockholm University Stockholm Sweden
BackLink https://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-225644$$DView record from Swedish Publication Index
BookMark eNpdkV1LwzAUhoNMcJuCP6HgjRd2niRN01yO-TFh4I16G7I2mRltU5PW4b83ZaLguTkceHh5Du8MTVrXaoQuMSwwALkNtlmIomAnaIpB8BQIKyZoCoTzNOeYnaFZCHsAjBnhU7ReOe912dt2l3Ref6pat6VOdOhto3rr2sQ4n2ytCrpKgmq6eiQPtn9P-hGKh_be-XCOTo2qg7742XP0-nD_slqnm-fHp9Vyk5aUiT5VUDGgOcGkolgxylQcU_GKC2EE0C0DXWScAtMiyzNDqSlzQbhhuCSYKTpHN8fccNDdsJWdj6L-Szpl5Z19W0rndzIMkhCWZ1nEr494593HEI1lY0Op61q12g1BkiKHDBilLKJX_9C9G3wbn4kU56yAKPAXWHoXgtfm1wCDHBuQsQE5NkC_AZ1Hegs
CitedBy_id crossref_primary_10_3847_1538_4365_ad2c88
crossref_primary_10_1109_TIT_2023_3327399
Cites_doi 10.1093/oxfordjournals.aje.a112510
10.5048/BIO‐C.2018.4
10.3390/e24101469
10.1186/s12874‐020‐01081‐0
10.5048/BIO‐C.2020.4
10.1016/j.jtbi.2020.110556
10.3390/e24101323
10.1109/ICSMC.2009.5346119
10.1007/978-3-540-87987-9_8
10.1103/PhysRev.106.620
10.1002/rmv.2200
10.1016/S2214‐109X(20)30074‐7
10.1155/2011/608719
10.1038/s41746‐020‐00372‐6
10.1109/TSMC.2021.3056669
10.20965/jaciii.2010.p0475
10.1038/s41467‐021‐22082‐7
10.1109/TSMCA.2009.2025027
10.1109/SMC.2017.8122651
10.1371/journal.pone.0242958
10.1093/jssam/smac029
10.5048/BIO‐C.2020.3
10.1016/j.spl.2020.108742
10.3899/jrheum.130675
10.1109/TIT.2023.3327399
10.1109/4235.585893
10.1257/aer.20180310
10.1016/S2589‐7500(20)30133‐3
10.1002/asmb.2430
10.1109/TPAMI.2022.3195462
10.1007/s10701‐022‐00650‐1
10.1002/j.1538-7305.1948.tb01338.x
10.1016/j.epidem.2018.01.002
10.1088/1475‐7516/2021/07/020
ContentType Journal Article
Copyright 2023. This article is published under http://creativecommons.org/licenses/by-nc/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2023. This article is published under http://creativecommons.org/licenses/by-nc/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID AAYXX
CITATION
K9.
7X8
ABAVF
ADTPV
AOWAS
D8T
DG7
ZZAVC
DOI 10.1002/sim.9885
DatabaseName CrossRef
ProQuest Health & Medical Complete (Alumni)
MEDLINE - Academic
SWEPUB Stockholms universitet full text
SwePub
SwePub Articles
SWEPUB Freely available online
SWEPUB Stockholms universitet
SwePub Articles full text
DatabaseTitle CrossRef
ProQuest Health & Medical Complete (Alumni)
MEDLINE - Academic
DatabaseTitleList ProQuest Health & Medical Complete (Alumni)
CrossRef
DeliveryMethod fulltext_linktorsrc
Discipline Medicine
Statistics
Public Health
EISSN 1097-0258
EndPage 4737
ExternalDocumentID oai_DiVA_org_su_225644
10_1002_sim_9885
GroupedDBID ---
.3N
.GA
05W
0R~
10A
123
1L6
1OB
1OC
1ZS
33P
3SF
3WU
4.4
4ZD
50Y
50Z
51W
51X
52M
52N
52O
52P
52S
52T
52U
52W
52X
5RE
5VS
66C
6PF
702
7PT
8-0
8-1
8-3
8-4
8-5
8UM
930
A03
AAESR
AAEVG
AAHHS
AANLZ
AAONW
AAWTL
AAXRX
AAYXX
AAZKR
ABCQN
ABCUV
ABIJN
ABJNI
ABOCM
ABPVW
ACAHQ
ACCFJ
ACCZN
ACGFS
ACPOU
ACXBN
ACXQS
ADBBV
ADEOM
ADIZJ
ADKYN
ADMGS
ADOZA
ADXAS
ADZMN
AEEZP
AEIGN
AEIMD
AENEX
AEQDE
AEUQT
AEUYR
AFBPY
AFFPM
AFGKR
AFPWT
AFZJQ
AHBTC
AHMBA
AITYG
AIURR
AIWBW
AJBDE
AJXKR
ALAGY
ALMA_UNASSIGNED_HOLDINGS
ALUQN
AMBMR
AMYDB
ATUGU
AUFTA
AZBYB
AZVAB
BAFTC
BFHJK
BHBCM
BMNLL
BMXJE
BNHUX
BROTX
BRXPI
BY8
CITATION
CS3
D-E
D-F
DCZOG
DPXWK
DR2
DRFUL
DRSTM
DU5
EBS
F00
F01
F04
F5P
G-S
G.N
GNP
GODZA
H.T
H.X
HBH
HGLYW
HHY
HHZ
HZ~
IX1
J0M
JPC
KQQ
LATKE
LAW
LC2
LC3
LEEKS
LH4
LITHE
LOXES
LP6
LP7
LUTES
LYRES
MEWTI
MK4
MRFUL
MRSTM
MSFUL
MSSTM
MXFUL
MXSTM
N04
N05
N9A
NF~
NNB
O66
O9-
OIG
P2P
P2W
P2X
P4D
PALCI
PQQKQ
Q.N
Q11
QB0
QRW
R.K
ROL
RWI
RX1
RYL
SUPJJ
TN5
UB1
V2E
W8V
W99
WBKPD
WH7
WIB
WIH
WIK
WJL
WOHZO
WQJ
WRC
WUP
WWH
WXSBR
WYISQ
XBAML
XG1
XV2
ZZTAW
~IA
~WT
K9.
7X8
.Y3
31~
53G
AASGY
ABAVF
ABEML
ACBWZ
ACSCC
ADTPV
AFFNX
AOWAS
ASPBG
AVWKF
AZFZN
BDRZF
D8T
DG7
DUUFO
EBD
EJD
EMOBN
EX3
FEDTE
HF~
HVGLF
LW6
M67
RIWAO
RJQFR
SAMSI
SV3
WOW
YHZ
ZGI
ZXP
ZZAVC
ID FETCH-LOGICAL-c359t-a0d5036212d31a535aaaafd7d799f903b50e847305e9464f33fc6927f51c215a3
ISSN 0277-6715
1097-0258
IngestDate Fri Aug 23 23:56:14 EDT 2024
Fri Aug 16 01:26:39 EDT 2024
Thu Oct 10 20:19:14 EDT 2024
Fri Aug 23 02:23:40 EDT 2024
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 26
Language English
LinkModel OpenURL
MergedId FETCHMERGED-LOGICAL-c359t-a0d5036212d31a535aaaafd7d799f903b50e847305e9464f33fc6927f51c215a3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ORCID 0000-0001-6281-1720
OpenAccessLink https://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-225644
PQID 2877580215
PQPubID 48361
PageCount 25
ParticipantIDs swepub_primary_oai_DiVA_org_su_225644
proquest_miscellaneous_2860405335
proquest_journals_2877580215
crossref_primary_10_1002_sim_9885
PublicationCentury 2000
PublicationDate 2023-11-20
PublicationDateYYYYMMDD 2023-11-20
PublicationDate_xml – month: 11
  year: 2023
  text: 2023-11-20
  day: 20
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle Statistics in medicine
PublicationYear 2023
Publisher Wiley Subscription Services, Inc
Publisher_xml – name: Wiley Subscription Services, Inc
References e_1_2_12_4_1
e_1_2_12_3_1
e_1_2_12_6_1
e_1_2_12_5_1
e_1_2_12_19_1
e_1_2_12_18_1
e_1_2_12_2_1
e_1_2_12_17_1
e_1_2_12_16_1
e_1_2_12_38_1
Carabaña JM (e_1_2_12_26_1) 2020; 94
e_1_2_12_20_1
e_1_2_12_41_1
e_1_2_12_21_1
Cover TM (e_1_2_12_39_1) 2006
e_1_2_12_22_1
e_1_2_12_23_1
e_1_2_12_24_1
e_1_2_12_25_1
e_1_2_12_40_1
e_1_2_12_27_1
e_1_2_12_28_1
e_1_2_12_29_1
Barbier J (e_1_2_12_12_1) 2020
e_1_2_12_30_1
e_1_2_12_31_1
e_1_2_12_32_1
e_1_2_12_33_1
e_1_2_12_34_1
e_1_2_12_35_1
e_1_2_12_36_1
e_1_2_12_37_1
e_1_2_12_15_1
e_1_2_12_14_1
e_1_2_12_13_1
e_1_2_12_8_1
e_1_2_12_11_1
e_1_2_12_7_1
e_1_2_12_10_1
e_1_2_12_9_1
References_xml – ident: e_1_2_12_24_1
  doi: 10.1093/oxfordjournals.aje.a112510
– ident: e_1_2_12_34_1
  doi: 10.5048/BIO‐C.2018.4
– start-page: 99
  year: 2020
  ident: e_1_2_12_12_1
  article-title: High‐dimensional inference: a statistical mechanics perspective
  publication-title: Ithaca Viaggio Nella Sci
  contributor:
    fullname: Barbier J
– ident: e_1_2_12_13_1
  doi: 10.3390/e24101469
– ident: e_1_2_12_25_1
  doi: 10.1186/s12874‐020‐01081‐0
– ident: e_1_2_12_38_1
  doi: 10.5048/BIO‐C.2020.4
– ident: e_1_2_12_11_1
  doi: 10.1016/j.jtbi.2020.110556
– ident: e_1_2_12_41_1
  doi: 10.3390/e24101323
– ident: e_1_2_12_30_1
  doi: 10.1109/ICSMC.2009.5346119
– volume: 94
  year: 2020
  ident: e_1_2_12_26_1
  article-title: Datos de encuesta Para estimar la prevalencia de COVID‐19. Un estudio piloto en Madrid capital
  publication-title: Rev Esp Salud Publica
  contributor:
    fullname: Carabaña JM
– ident: e_1_2_12_9_1
  doi: 10.1007/978-3-540-87987-9_8
– ident: e_1_2_12_17_1
  doi: 10.1103/PhysRev.106.620
– ident: e_1_2_12_27_1
  doi: 10.1002/rmv.2200
– ident: e_1_2_12_4_1
  doi: 10.1016/S2214‐109X(20)30074‐7
– ident: e_1_2_12_23_1
  doi: 10.1155/2011/608719
– ident: e_1_2_12_19_1
  doi: 10.1038/s41746‐020‐00372‐6
– volume-title: Elements of Information Theory
  year: 2006
  ident: e_1_2_12_39_1
  contributor:
    fullname: Cover TM
– ident: e_1_2_12_20_1
– ident: e_1_2_12_22_1
  doi: 10.1109/TSMC.2021.3056669
– ident: e_1_2_12_32_1
  doi: 10.20965/jaciii.2010.p0475
– ident: e_1_2_12_8_1
– ident: e_1_2_12_5_1
  doi: 10.1038/s41467‐021‐22082‐7
– ident: e_1_2_12_31_1
  doi: 10.1109/TSMCA.2009.2025027
– ident: e_1_2_12_33_1
  doi: 10.1109/SMC.2017.8122651
– ident: e_1_2_12_21_1
  doi: 10.1371/journal.pone.0242958
– ident: e_1_2_12_10_1
  doi: 10.1093/jssam/smac029
– ident: e_1_2_12_18_1
  doi: 10.5048/BIO‐C.2020.3
– ident: e_1_2_12_40_1
  doi: 10.1016/j.spl.2020.108742
– ident: e_1_2_12_29_1
  doi: 10.3899/jrheum.130675
– ident: e_1_2_12_28_1
  doi: 10.1109/TIT.2023.3327399
– ident: e_1_2_12_35_1
  doi: 10.1109/4235.585893
– ident: e_1_2_12_7_1
  doi: 10.1257/aer.20180310
– ident: e_1_2_12_6_1
  doi: 10.1016/S2589‐7500(20)30133‐3
– ident: e_1_2_12_36_1
  doi: 10.1002/asmb.2430
– ident: e_1_2_12_37_1
  doi: 10.1109/TPAMI.2022.3195462
– ident: e_1_2_12_16_1
  doi: 10.1007/s10701‐022‐00650‐1
– ident: e_1_2_12_3_1
– ident: e_1_2_12_14_1
  doi: 10.1002/j.1538-7305.1948.tb01338.x
– ident: e_1_2_12_2_1
  doi: 10.1016/j.epidem.2018.01.002
– ident: e_1_2_12_15_1
  doi: 10.1088/1475‐7516/2021/07/020
SSID ssj0011527
Score 2.482842
Snippet Sampling for prevalence estimation of infection is subject to bias by both oversampling of symptomatic individuals and error‐prone tests. This results in naïve...
Sampling for prevalence estimation of infection is subject to bias by both oversampling of symptomatic individuals and error-prone tests. This results in naïve...
SourceID swepub
proquest
crossref
SourceType Open Access Repository
Aggregation Database
StartPage 4713
SubjectTerms active information
bias correction
COVID-19
maximum entropy
prevalence
sampling
sampling bias
testing errors
Title Correcting prevalence estimation for biased sampling with testing errors
URI https://www.proquest.com/docview/2877580215
https://search.proquest.com/docview/2860405335
https://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-225644
Volume 42
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1bb9MwFLbKkNAkhKCAGAxkJHiqUtI4l_pxtJsqtA00tajai-XEzhppS1CbvOw38KM5J3YuaAgx-hClbuRUPp_PxT7nMyEflB-mU5nARPJ15PiKp840lIGjsEzTVTHX9dmAZ-fhYuV_WQfrweBnL2upKuNxcvvHupL_kSq0gVyxSvYekm07hQa4B_nCFSQM13-S8QyP1khKU1COvN31NEXejJsuhzDOwFCp0U5i7ni78ooPwRe93RZ2P8d6qOh9WvLmLL-z9X65KSoTyl9n3SI3VqnXqZFm2303mtc3c3nb5lJ8k8kGGz-zFo0zWxhyuZFFm8gDmizPrkcXtsmuR3gMC_M89x5ar6fhcP84jEw951gbDewiPaxn-NwbFe17PSh6fYULtpX1jLcfGQqZO4bBEM3uspsxn06Dzvg1G_7nX8XJ6vRULI_XywfkoQdqCxME5xctGdmkOQC4-dMNlbHrfWr6_d256UUsfRba2nNZPiVPbMhBjwx-npGBzofk0ZmV7JA8Nku31FSkDcl-B4HnZNFhjHYYox3GKGCMGozRBmMUMUYtxqjB2AuyOjlezhaOPX3DSVjAS0e6KkD3ZuIpNpEBCyR8UhWpiPOUuywOXA2uDdgLzf3QTxlLk5B7URpMEvAjJXtJ9vIi168IjcEpDGFI0X31VRJKrWKsuFY8jmPo7YC8b4ZN_DAkK8LQaXsChlbg0B6Qw2Y8hZ2COwHhPsS76LZCF-3PoCBx10vmuqjwmRAMFUQ18MxHI4f2JcitPs--H4lieyV2lQDjBvHB67-_6g3Z73B_SPbKbaXfgmdaxu9qwPwCBTKP4Q
link.rule.ids 230,315,783,787,888,27938,27939
linkProvider Wiley-Blackwell
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Correcting+prevalence+estimation+for+biased+sampling+with+testing+errors&rft.jtitle=Statistics+in+medicine&rft.au=Zhou%2C+Lili&rft.au=Daniel+Andr%C3%A9s+D%C3%ADaz%E2%80%90Pach%C3%B3n&rft.au=Chen%2C+Zhao&rft.au=J+Sunil+Rao&rft.date=2023-11-20&rft.pub=Wiley+Subscription+Services%2C+Inc&rft.issn=0277-6715&rft.eissn=1097-0258&rft.volume=42&rft.issue=26&rft.spage=4713&rft.epage=4737&rft_id=info:doi/10.1002%2Fsim.9885&rft.externalDBID=NO_FULL_TEXT
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0277-6715&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0277-6715&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0277-6715&client=summon