Delphi, non-RAND modified Delphi, RAND/UCLA appropriateness method and a novel group awareness and consensus methodology for consensus measurement: a systematic literature review

Increasing demand for reliable evidence in patient care and its delivery has necessitated the development of several approaches for generating quality evidence. In particular, the solicitation of expert opinion has been recognised as a reliable data collection method. However, there are variations a...

Full description

Saved in:
Bibliographic Details
Published inCurrent medical research and opinion Vol. 36; no. 11; pp. 1873 - 1887
Main Author Jandhyala, Ravi
Format Journal Article
LanguageEnglish
Published England Taylor & Francis 01.11.2020
Subjects
Online AccessGet full text
ISSN0300-7995
1473-4877
DOI10.1080/03007995.2020.1816946

Cover

Abstract Increasing demand for reliable evidence in patient care and its delivery has necessitated the development of several approaches for generating quality evidence. In particular, the solicitation of expert opinion has been recognised as a reliable data collection method. However, there are variations and limitations in study approaches using expert opinion as a method of data collection, thereby necessitating the development of a standardised, novel consensus method. A systematic literature review was conducted to assess the characteristics of all studies utilising a "Delphi" or "Modified Delphi" methods between January 2008 and December 2018. A search framework was developed, and the review was conducted in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. In total, 764 studies met the study inclusion and exclusion criteria and were included in the review. Heterogeneity on core defining characteristics of the constituent study types was observed in this control group. These 764 studies were compared against the four case studies using the Jandhyala method. Four key themes were identified and discussed: Assessment of Forced or Observed consensus, Assessment and reporting of item Awareness and advisor awareness, Minimum expert engagement profile, and Efficiency of Minimum Engagement Profile. Existing consensus methodologies have undergone significant modifications by successive authors over time, including ones contradicting core principles where an original method had been defined. The Jandhyala method for generating group consensus and awareness is unique in observing consensus and measuring awareness of subject matter across experts. The Jandhyala method also improves upon the traditional Delphi-style methodologies, through the introduction of new insights into awareness of subject matter in the expert group. A wider application of the Jandhyala method is required to corroborate findings from this research.
AbstractList Increasing demand for reliable evidence in patient care and its delivery has necessitated the development of several approaches for generating quality evidence. In particular, the solicitation of expert opinion has been recognised as a reliable data collection method. However, there are variations and limitations in study approaches using expert opinion as a method of data collection, thereby necessitating the development of a standardised, novel consensus method. A systematic literature review was conducted to assess the characteristics of all studies utilising a "Delphi" or "Modified Delphi" methods between January 2008 and December 2018. A search framework was developed, and the review was conducted in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. In total, 764 studies met the study inclusion and exclusion criteria and were included in the review. Heterogeneity on core defining characteristics of the constituent study types was observed in this control group. These 764 studies were compared against the four case studies using the Jandhyala method. Four key themes were identified and discussed: Assessment of Forced or Observed consensus, Assessment and reporting of item Awareness and advisor awareness, Minimum expert engagement profile, and Efficiency of Minimum Engagement Profile. Existing consensus methodologies have undergone significant modifications by successive authors over time, including ones contradicting core principles where an original method had been defined. The Jandhyala method for generating group consensus and awareness is unique in observing consensus and measuring awareness of subject matter across experts. The Jandhyala method also improves upon the traditional Delphi-style methodologies, through the introduction of new insights into awareness of subject matter in the expert group. A wider application of the Jandhyala method is required to corroborate findings from this research.
Author Jandhyala, Ravi
Author_xml – sequence: 1
  givenname: Ravi
  orcidid: 0000-0002-7241-7476
  surname: Jandhyala
  fullname: Jandhyala, Ravi
  organization: Medialis Ltd
BackLink https://www.ncbi.nlm.nih.gov/pubmed/32866051$$D View this record in MEDLINE/PubMed
BookMark eNpVkd1q3DAQhUVJaTZpHyFBD1AnkizrJ1ddNklbWFoIybXRWuNEwZaMpM2yr9UnjMxmL3oxDMz55sDMOUMnPnhA6IKSK0oUuSY1IVLr5ooRVkaKCs3FJ7SgXNYVV1KeoMXMVDN0is5SeiWEMqX1F3RaMyUEaegC_buFYXpx33Gxrx6Wf27xGKzrHVh8VObp9dNqvcRmmmKYojMZPKSER8gvwWLjSxWDNxjwcwzbCZudiQdk1rrgE_i0PS6EITzvcR_if4pJ2wgj-HxTzNI-ZRhNdh0eXIZochFxhDcHu6_oc2-GBN8--jl6ur97XP2q1n9__l4t11XHpc4V1UxwCwyo3DRK9oxtCCe0b6CTRkjFG2o5k8JIyxvRGQ2dZUI1fc9r1Qhdn6PLg--03Yxg23L4aOK-PT6vAD8OgPPlmNHsQhxsm81-CLGPxncutTUl7ZxXe8yrnfNqP_Kq3wFyD4tR
CitedBy_id crossref_primary_10_3389_fpubh_2024_1384498
crossref_primary_10_1136_bjsports_2021_104578
crossref_primary_10_1093_tbm_ibae020
crossref_primary_10_3390_geriatrics7050100
crossref_primary_10_1177_08919887221106446
crossref_primary_10_1590_1980_265x_tce_2023_0227en
crossref_primary_10_1001_jamaophthalmol_2023_5750
crossref_primary_10_1080_03007995_2023_2256222
crossref_primary_10_1111_anae_15808
crossref_primary_10_1590_1980_265x_tce_2023_0227es
crossref_primary_10_3389_fphar_2022_842431
crossref_primary_10_1159_000538072
crossref_primary_10_1186_s12955_021_01862_1
crossref_primary_10_1016_j_jcpo_2024_100515
crossref_primary_10_1080_03007995_2022_2144054
crossref_primary_10_1080_13506129_2021_1931099
crossref_primary_10_3390_biomedicines12081846
crossref_primary_10_3389_fpubh_2022_1004910
crossref_primary_10_1016_j_eclinm_2024_102918
crossref_primary_10_1080_03007995_2023_2165814
crossref_primary_10_1111_jocn_17505
crossref_primary_10_1093_cid_ciab111
crossref_primary_10_1186_s12913_023_09604_3
crossref_primary_10_3390_ijerph182010988
crossref_primary_10_1136_bmjopen_2024_086225
crossref_primary_10_1177_15357597241293310
crossref_primary_10_1016_j_clml_2024_10_013
crossref_primary_10_1136_bmjopen_2021_057797
crossref_primary_10_1136_bjsports_2023_107552
crossref_primary_10_1080_03007995_2024_2362278
crossref_primary_10_1002_hpja_941
crossref_primary_10_1080_03007995_2023_2172261
crossref_primary_10_1136_tsaco_2021_000805
crossref_primary_10_1177_17446295231225272
crossref_primary_10_1186_s43043_024_00171_z
crossref_primary_10_1002_ehf2_15153
crossref_primary_10_1002_alz_12461
crossref_primary_10_1080_03007995_2021_1947218
crossref_primary_10_3390_healthcare12202049
crossref_primary_10_3390_su16135607
crossref_primary_10_1371_journal_pone_0319620
crossref_primary_10_3390_medicina58050627
crossref_primary_10_1007_s40290_021_00413_9
crossref_primary_10_1016_j_semerg_2024_102217
crossref_primary_10_1515_edu_2022_0213
crossref_primary_10_1038_s41533_022_00276_w
crossref_primary_10_1007_s11739_023_03238_3
crossref_primary_10_1017_S1478951521002030
crossref_primary_10_1177_29767342241246762
crossref_primary_10_1590_1980_265x_tce_2023_0227pt
crossref_primary_10_1007_s40279_023_01948_4
crossref_primary_10_1002_mus_27941
crossref_primary_10_1080_03007995_2023_2284386
crossref_primary_10_1136_bjsports_2023_106898
crossref_primary_10_1111_papr_13176
crossref_primary_10_1016_j_jval_2024_07_027
crossref_primary_10_1016_j_parkreldis_2025_107295
crossref_primary_10_1111_1471_0528_17451
crossref_primary_10_1186_s12874_023_02087_0
crossref_primary_10_1111_bjd_19893
crossref_primary_10_1080_10749357_2022_2083296
crossref_primary_10_1002_ijgo_16118
crossref_primary_10_1080_03007995_2023_2235153
crossref_primary_10_1001_jamadermatol_2023_1347
crossref_primary_10_1080_03007995_2023_2174747
crossref_primary_10_1097_ADM_0000000000001369
crossref_primary_10_1016_j_ajic_2021_12_012
crossref_primary_10_2147_COPD_S481337
crossref_primary_10_1136_bmjopen_2024_087739
crossref_primary_10_1016_j_contraception_2025_110829
crossref_primary_10_1136_bmjopen_2023_072232
crossref_primary_10_1080_03007995_2021_1927689
ContentType Journal Article
Copyright 2020 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. 2020
Copyright_xml – notice: 2020 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. 2020
DBID 0YH
CGR
CUY
CVF
ECM
EIF
NPM
DOI 10.1080/03007995.2020.1816946
DatabaseName Taylor & Francis Open Access
Medline
MEDLINE
MEDLINE (Ovid)
MEDLINE
MEDLINE
PubMed
DatabaseTitle MEDLINE
Medline Complete
MEDLINE with Full Text
PubMed
MEDLINE (Ovid)
DatabaseTitleList MEDLINE

Database_xml – sequence: 1
  dbid: NPM
  name: PubMed
  url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
  sourceTypes: Index Database
– sequence: 2
  dbid: EIF
  name: MEDLINE
  url: https://proxy.k.utb.cz/login?url=https://www.webofscience.com/wos/medline/basic-search
  sourceTypes: Index Database
– sequence: 3
  dbid: 0YH
  name: Taylor & Francis Open Access
  url: https://www.tandfonline.com
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Medicine
EISSN 1473-4877
EndPage 1887
ExternalDocumentID 32866051
1816946
Genre Research Article
Research Support, Non-U.S. Gov't
Systematic Review
Journal Article
GroupedDBID ---
00X
03L
0BK
0R~
0YH
29F
4.4
5GY
5RE
6J9
6PF
AALUX
AAMIU
AAPUL
AAQRR
AAWTL
ABBKH
ABEIZ
ABIVO
ABJNI
ABLIJ
ABLJU
ABLKL
ABOCM
ABUPF
ABXYU
ACENM
ACGFO
ACGFS
ACGOD
ACIEZ
ADCVX
ADRBQ
AECIN
AENEX
AEOZL
AGDLA
AGFJD
AGRBW
AGYJP
AHMBA
AIJEM
AIRBT
AKBVH
ALIPV
ALMA_UNASSIGNED_HOLDINGS
ALQZU
ALYBC
AMDAE
BABNJ
BLEHA
BOHLJ
CCCUG
CS3
DKSSO
DU5
EBS
F5P
H13
HZ~
KRBQP
KSSTO
KWAYT
KYCEM
L7B
LJTGL
M4Z
O9-
P2P
RNANH
RVRKI
RWL
SJN
TAE
TBQAZ
TDBHL
TERGH
TFDNU
TFL
TFW
TUROJ
UEQFS
V1S
~1N
.55
.GJ
34G
39C
53G
7X7
88E
88I
8AF
8AO
8FI
8FJ
8R4
8R5
AALIY
AAORF
AAPXX
ABBAB
ABUWG
ABWCV
ABWVI
ABZEW
ACKZS
ACOPL
ADBBV
ADFOM
ADFZZ
ADYSH
AEIIZ
AFKRA
AFLEI
AFRVT
AJVHN
AMPGV
AZQEC
BENPR
BPHCQ
BRMBE
BVXVI
CAG
CCPQU
CGR
COF
CUY
CVF
CYYVM
CZDIS
DRXRE
DWQXO
DWTOO
ECM
EIF
EJD
EMOBN
FYUFA
GNUQQ
HCIFZ
HMCUK
IPNFZ
J5H
JENTW
M1P
M2P
M2Q
M44
NPM
NUSFT
PHGZM
PHGZT
PQQKQ
PROAC
PSQYO
Q2X
QQXMO
RIG
S0X
UKHRP
X7M
ZGI
ID FETCH-LOGICAL-c479t-19264de2e17b587f22b0401f5ec7a678451d4276a7d456ca9ecd2685ff4385693
IEDL.DBID 0YH
ISSN 0300-7995
IngestDate Sun May 11 04:07:59 EDT 2025
Wed Dec 25 09:07:18 EST 2024
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 11
Keywords case studies
expert opinion
Delphi
Jandhyala method
novel consensus method
Language English
License open-access: http://creativecommons.org/licenses/by-nc-nd/4.0/: This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c479t-19264de2e17b587f22b0401f5ec7a678451d4276a7d456ca9ecd2685ff4385693
ORCID 0000-0002-7241-7476
OpenAccessLink https://www.tandfonline.com/doi/abs/10.1080/03007995.2020.1816946
PMID 32866051
PageCount 15
ParticipantIDs informaworld_taylorfrancis_310_1080_03007995_2020_1816946
pubmed_primary_32866051
PublicationCentury 2000
PublicationDate 2020-11-01
PublicationDateYYYYMMDD 2020-11-01
PublicationDate_xml – month: 11
  year: 2020
  text: 2020-11-01
  day: 01
PublicationDecade 2020
PublicationPlace England
PublicationPlace_xml – name: England
PublicationTitle Current medical research and opinion
PublicationTitleAlternate Curr Med Res Opin
PublicationYear 2020
Publisher Taylor & Francis
Publisher_xml – name: Taylor & Francis
SSID ssj0012899
Score 2.553145
SecondaryResourceType review_article
Snippet Increasing demand for reliable evidence in patient care and its delivery has necessitated the development of several approaches for generating quality...
SourceID pubmed
informaworld
SourceType Index Database
Publisher
StartPage 1873
SubjectTerms case studies
Consensus
Delivery of Health Care
Delphi
Delphi Technique
expert opinion
Humans
Jandhyala method
novel consensus method
Research Design
Title Delphi, non-RAND modified Delphi, RAND/UCLA appropriateness method and a novel group awareness and consensus methodology for consensus measurement: a systematic literature review
URI https://www.tandfonline.com/doi/abs/10.1080/03007995.2020.1816946
https://www.ncbi.nlm.nih.gov/pubmed/32866051
Volume 36
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1bS8MwFA46QXwR784befDRsCZN09a3sTmGuCHiQJ9KkyYozDlcp__LX-hJ0k4d-ORLobm1nJPLd5JzviB0LnOR8iiPiaFaEm5EQJJEhMRwrmA1oUq5KNfBUPRH_Pohqr0JZ5VbpbWhjSeKcHO1Hdy5nNUecS3ol4HlMQPrjkFSQuFzYhWtMQCK1qsveOwvDhKsPeEPEgJi69RBPH81s8RYugQ23aLT20KbFVrEba_ebbSiJztofVCdh--iz64eT5-eLzAY8eSuPezil9fi2QCsxHWOTW2NOjdt7OjDp9DhSje_YX95NAYx4BwaeNdj7GI8cP5hI8RsEZunrL_1ZDavK7h9eAw__ytnsdN4CY1900Pj8YK2GfsomT006l3dd_qkuoWBKB6nJQEIKHihmaaxjJLYMCZh4FMTaRXnsNTxiBacxSKPCwBjKk-1KphIImN4mEQiDfdRA4SgDxEOwTbh2mgw2hSXLEwllCgYiJ2mEpBGE6U_hZ-VbtPC-BtGspDWdKaV4jKruKxSXBMdeE1lU0_TkYUsEWCs0aN_tHqMNuyrDz48QY3yba5PAYWU8sz1M3gObwdfsF7WMg
linkProvider Taylor & Francis
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LT8MwDI5gSMAF8WY8c-BItCVN05bbtIEGbDugTRqnqk0TMWmMCTr4X_xCnKQdYxInrnHiVraT-HNiB6HLNBER95OAaKpSwrWokzAUHtGcS9hNqJQ2y7XbE-0Bvx_6w4VcGHOt0mBo7QpF2LXaTG4TjC6vxNXAMOumkBnAOwZNIYXviVW05oeAJsCm60_t-UmCARTuJKFOzJgyi-cvNkslS5e8Tbvr3G6jrcJdxA2n3x20oia7aL1bHIjvoa-WGk-fR1cYUDx5bPRa-OU1G2nwK3FJMa21QbPTwLZ--BQsLrcLHHavR2OQA06AwYcaY5vkgZNPkyJmuhiaNBeuJ--zcoANxGP4-V-UeajxGpj91IfG43ndZuzSZPbR4Pam32yT4hkGInkQ5QR8QMEzxRQNUj8MNGMpzHyqfSWDBPY67tOMs0AkQQbemEwiJTMmQl9r7oW-iLwDVAEhqCOEPQAnXGkFqE3ylHlRCj0yBmKnUQquRhVFi8KPcxu10O6JkdijZT3TQnGxUVxcKK6KDp2m4qmr0xF7LBSA1ujxP7heoI12v9uJO3e9hxO0aUguE_EUVfK3mToDlyRPz63NfQM7p9iR
linkToPdf http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3JTsMwELWgSBUXxE5ZfeCI1dpxnIRb1VIVaCuEqASnKHFsUamUiqbwX3whYzspUIkTVy-TaMbLG3vmGaHzNBER95OAaKpSwrVokDAUHtGcS9hNqJQ2y7U_EN0hv3n0y2jCWRFWaXxo7Ygi7FptJvc002VEXB3GZcPwmIF3x6AopPA5sYrW_DCyGKnx1F1cJBh_wl0kNIjpUybx_CVmibF0CWzaTaeziTYKtIibzrxbaEVNtlG1X9yH76DPthpPn0cXGJx4ct8ctPHLazbSACtxWWNK68NWr4ktffgUBlxu1zfsHo_GoAacgIB3NcY2xwMnHyZDzDQxddLEW09m87KDPYfH8PO_ahYnjZcg7JseGo8XtM3YZcnsomHn6qHVJcUrDETyIMoJQEDBM8UUDVI_DDRjKUx8qn0lgwS2Ou7TjLNAJEEGYEwmkZIZE6GvNfdCX0TeHqqAEtQBwh74JlxpBU6b5CnzohRaZAzUTqMUkEYNRT-VH-f20EK7F0Zij5Z0poXhYmO4uDBcDe07S8VTR9MReywU4KzRw39IPUPVu3Yn7l0Pbo_QuqlxeYjHqJK_zdUJAJI8PbVD7gsAw9e6
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Delphi%2C+non-RAND+modified+Delphi%2C+RAND%2FUCLA+appropriateness+method+and+a+novel+group+awareness+and+consensus+methodology+for+consensus+measurement%3A+a+systematic+literature+review&rft.jtitle=Current+medical+research+and+opinion&rft.au=Jandhyala%2C+Ravi&rft.date=2020-11-01&rft.pub=Taylor+%26+Francis&rft.issn=0300-7995&rft.eissn=1473-4877&rft.volume=36&rft.issue=11&rft.spage=1873&rft.epage=1887&rft_id=info:doi/10.1080%2F03007995.2020.1816946&rft.externalDBID=0YH&rft.externalDocID=1816946
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0300-7995&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0300-7995&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0300-7995&client=summon