Delphi, non-RAND modified Delphi, RAND/UCLA appropriateness method and a novel group awareness and consensus methodology for consensus measurement: a systematic literature review
Increasing demand for reliable evidence in patient care and its delivery has necessitated the development of several approaches for generating quality evidence. In particular, the solicitation of expert opinion has been recognised as a reliable data collection method. However, there are variations a...
Saved in:
Published in | Current medical research and opinion Vol. 36; no. 11; pp. 1873 - 1887 |
---|---|
Main Author | |
Format | Journal Article |
Language | English |
Published |
England
Taylor & Francis
01.11.2020
|
Subjects | |
Online Access | Get full text |
ISSN | 0300-7995 1473-4877 |
DOI | 10.1080/03007995.2020.1816946 |
Cover
Abstract | Increasing demand for reliable evidence in patient care and its delivery has necessitated the development of several approaches for generating quality evidence. In particular, the solicitation of expert opinion has been recognised as a reliable data collection method. However, there are variations and limitations in study approaches using expert opinion as a method of data collection, thereby necessitating the development of a standardised, novel consensus method.
A systematic literature review was conducted to assess the characteristics of all studies utilising a "Delphi" or "Modified Delphi" methods between January 2008 and December 2018. A search framework was developed, and the review was conducted in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.
In total, 764 studies met the study inclusion and exclusion criteria and were included in the review. Heterogeneity on core defining characteristics of the constituent study types was observed in this control group. These 764 studies were compared against the four case studies using the Jandhyala method. Four key themes were identified and discussed: Assessment of Forced or Observed consensus, Assessment and reporting of item Awareness and advisor awareness, Minimum expert engagement profile, and Efficiency of Minimum Engagement Profile.
Existing consensus methodologies have undergone significant modifications by successive authors over time, including ones contradicting core principles where an original method had been defined. The Jandhyala method for generating group consensus and awareness is unique in observing consensus and measuring awareness of subject matter across experts. The Jandhyala method also improves upon the traditional Delphi-style methodologies, through the introduction of new insights into awareness of subject matter in the expert group. A wider application of the Jandhyala method is required to corroborate findings from this research. |
---|---|
AbstractList | Increasing demand for reliable evidence in patient care and its delivery has necessitated the development of several approaches for generating quality evidence. In particular, the solicitation of expert opinion has been recognised as a reliable data collection method. However, there are variations and limitations in study approaches using expert opinion as a method of data collection, thereby necessitating the development of a standardised, novel consensus method.
A systematic literature review was conducted to assess the characteristics of all studies utilising a "Delphi" or "Modified Delphi" methods between January 2008 and December 2018. A search framework was developed, and the review was conducted in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.
In total, 764 studies met the study inclusion and exclusion criteria and were included in the review. Heterogeneity on core defining characteristics of the constituent study types was observed in this control group. These 764 studies were compared against the four case studies using the Jandhyala method. Four key themes were identified and discussed: Assessment of Forced or Observed consensus, Assessment and reporting of item Awareness and advisor awareness, Minimum expert engagement profile, and Efficiency of Minimum Engagement Profile.
Existing consensus methodologies have undergone significant modifications by successive authors over time, including ones contradicting core principles where an original method had been defined. The Jandhyala method for generating group consensus and awareness is unique in observing consensus and measuring awareness of subject matter across experts. The Jandhyala method also improves upon the traditional Delphi-style methodologies, through the introduction of new insights into awareness of subject matter in the expert group. A wider application of the Jandhyala method is required to corroborate findings from this research. |
Author | Jandhyala, Ravi |
Author_xml | – sequence: 1 givenname: Ravi orcidid: 0000-0002-7241-7476 surname: Jandhyala fullname: Jandhyala, Ravi organization: Medialis Ltd |
BackLink | https://www.ncbi.nlm.nih.gov/pubmed/32866051$$D View this record in MEDLINE/PubMed |
BookMark | eNpVkd1q3DAQhUVJaTZpHyFBD1AnkizrJ1ddNklbWFoIybXRWuNEwZaMpM2yr9UnjMxmL3oxDMz55sDMOUMnPnhA6IKSK0oUuSY1IVLr5ooRVkaKCs3FJ7SgXNYVV1KeoMXMVDN0is5SeiWEMqX1F3RaMyUEaegC_buFYXpx33Gxrx6Wf27xGKzrHVh8VObp9dNqvcRmmmKYojMZPKSER8gvwWLjSxWDNxjwcwzbCZudiQdk1rrgE_i0PS6EITzvcR_if4pJ2wgj-HxTzNI-ZRhNdh0eXIZochFxhDcHu6_oc2-GBN8--jl6ur97XP2q1n9__l4t11XHpc4V1UxwCwyo3DRK9oxtCCe0b6CTRkjFG2o5k8JIyxvRGQ2dZUI1fc9r1Qhdn6PLg--03Yxg23L4aOK-PT6vAD8OgPPlmNHsQhxsm81-CLGPxncutTUl7ZxXe8yrnfNqP_Kq3wFyD4tR |
CitedBy_id | crossref_primary_10_3389_fpubh_2024_1384498 crossref_primary_10_1136_bjsports_2021_104578 crossref_primary_10_1093_tbm_ibae020 crossref_primary_10_3390_geriatrics7050100 crossref_primary_10_1177_08919887221106446 crossref_primary_10_1590_1980_265x_tce_2023_0227en crossref_primary_10_1001_jamaophthalmol_2023_5750 crossref_primary_10_1080_03007995_2023_2256222 crossref_primary_10_1111_anae_15808 crossref_primary_10_1590_1980_265x_tce_2023_0227es crossref_primary_10_3389_fphar_2022_842431 crossref_primary_10_1159_000538072 crossref_primary_10_1186_s12955_021_01862_1 crossref_primary_10_1016_j_jcpo_2024_100515 crossref_primary_10_1080_03007995_2022_2144054 crossref_primary_10_1080_13506129_2021_1931099 crossref_primary_10_3390_biomedicines12081846 crossref_primary_10_3389_fpubh_2022_1004910 crossref_primary_10_1016_j_eclinm_2024_102918 crossref_primary_10_1080_03007995_2023_2165814 crossref_primary_10_1111_jocn_17505 crossref_primary_10_1093_cid_ciab111 crossref_primary_10_1186_s12913_023_09604_3 crossref_primary_10_3390_ijerph182010988 crossref_primary_10_1136_bmjopen_2024_086225 crossref_primary_10_1177_15357597241293310 crossref_primary_10_1016_j_clml_2024_10_013 crossref_primary_10_1136_bmjopen_2021_057797 crossref_primary_10_1136_bjsports_2023_107552 crossref_primary_10_1080_03007995_2024_2362278 crossref_primary_10_1002_hpja_941 crossref_primary_10_1080_03007995_2023_2172261 crossref_primary_10_1136_tsaco_2021_000805 crossref_primary_10_1177_17446295231225272 crossref_primary_10_1186_s43043_024_00171_z crossref_primary_10_1002_ehf2_15153 crossref_primary_10_1002_alz_12461 crossref_primary_10_1080_03007995_2021_1947218 crossref_primary_10_3390_healthcare12202049 crossref_primary_10_3390_su16135607 crossref_primary_10_1371_journal_pone_0319620 crossref_primary_10_3390_medicina58050627 crossref_primary_10_1007_s40290_021_00413_9 crossref_primary_10_1016_j_semerg_2024_102217 crossref_primary_10_1515_edu_2022_0213 crossref_primary_10_1038_s41533_022_00276_w crossref_primary_10_1007_s11739_023_03238_3 crossref_primary_10_1017_S1478951521002030 crossref_primary_10_1177_29767342241246762 crossref_primary_10_1590_1980_265x_tce_2023_0227pt crossref_primary_10_1007_s40279_023_01948_4 crossref_primary_10_1002_mus_27941 crossref_primary_10_1080_03007995_2023_2284386 crossref_primary_10_1136_bjsports_2023_106898 crossref_primary_10_1111_papr_13176 crossref_primary_10_1016_j_jval_2024_07_027 crossref_primary_10_1016_j_parkreldis_2025_107295 crossref_primary_10_1111_1471_0528_17451 crossref_primary_10_1186_s12874_023_02087_0 crossref_primary_10_1111_bjd_19893 crossref_primary_10_1080_10749357_2022_2083296 crossref_primary_10_1002_ijgo_16118 crossref_primary_10_1080_03007995_2023_2235153 crossref_primary_10_1001_jamadermatol_2023_1347 crossref_primary_10_1080_03007995_2023_2174747 crossref_primary_10_1097_ADM_0000000000001369 crossref_primary_10_1016_j_ajic_2021_12_012 crossref_primary_10_2147_COPD_S481337 crossref_primary_10_1136_bmjopen_2024_087739 crossref_primary_10_1016_j_contraception_2025_110829 crossref_primary_10_1136_bmjopen_2023_072232 crossref_primary_10_1080_03007995_2021_1927689 |
ContentType | Journal Article |
Copyright | 2020 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. 2020 |
Copyright_xml | – notice: 2020 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. 2020 |
DBID | 0YH CGR CUY CVF ECM EIF NPM |
DOI | 10.1080/03007995.2020.1816946 |
DatabaseName | Taylor & Francis Open Access Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed |
DatabaseTitle | MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) |
DatabaseTitleList | MEDLINE |
Database_xml | – sequence: 1 dbid: NPM name: PubMed url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: EIF name: MEDLINE url: https://proxy.k.utb.cz/login?url=https://www.webofscience.com/wos/medline/basic-search sourceTypes: Index Database – sequence: 3 dbid: 0YH name: Taylor & Francis Open Access url: https://www.tandfonline.com sourceTypes: Publisher |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Medicine |
EISSN | 1473-4877 |
EndPage | 1887 |
ExternalDocumentID | 32866051 1816946 |
Genre | Research Article Research Support, Non-U.S. Gov't Systematic Review Journal Article |
GroupedDBID | --- 00X 03L 0BK 0R~ 0YH 29F 4.4 5GY 5RE 6J9 6PF AALUX AAMIU AAPUL AAQRR AAWTL ABBKH ABEIZ ABIVO ABJNI ABLIJ ABLJU ABLKL ABOCM ABUPF ABXYU ACENM ACGFO ACGFS ACGOD ACIEZ ADCVX ADRBQ AECIN AENEX AEOZL AGDLA AGFJD AGRBW AGYJP AHMBA AIJEM AIRBT AKBVH ALIPV ALMA_UNASSIGNED_HOLDINGS ALQZU ALYBC AMDAE BABNJ BLEHA BOHLJ CCCUG CS3 DKSSO DU5 EBS F5P H13 HZ~ KRBQP KSSTO KWAYT KYCEM L7B LJTGL M4Z O9- P2P RNANH RVRKI RWL SJN TAE TBQAZ TDBHL TERGH TFDNU TFL TFW TUROJ UEQFS V1S ~1N .55 .GJ 34G 39C 53G 7X7 88E 88I 8AF 8AO 8FI 8FJ 8R4 8R5 AALIY AAORF AAPXX ABBAB ABUWG ABWCV ABWVI ABZEW ACKZS ACOPL ADBBV ADFOM ADFZZ ADYSH AEIIZ AFKRA AFLEI AFRVT AJVHN AMPGV AZQEC BENPR BPHCQ BRMBE BVXVI CAG CCPQU CGR COF CUY CVF CYYVM CZDIS DRXRE DWQXO DWTOO ECM EIF EJD EMOBN FYUFA GNUQQ HCIFZ HMCUK IPNFZ J5H JENTW M1P M2P M2Q M44 NPM NUSFT PHGZM PHGZT PQQKQ PROAC PSQYO Q2X QQXMO RIG S0X UKHRP X7M ZGI |
ID | FETCH-LOGICAL-c479t-19264de2e17b587f22b0401f5ec7a678451d4276a7d456ca9ecd2685ff4385693 |
IEDL.DBID | 0YH |
ISSN | 0300-7995 |
IngestDate | Sun May 11 04:07:59 EDT 2025 Wed Dec 25 09:07:18 EST 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | true |
IsScholarly | true |
Issue | 11 |
Keywords | case studies expert opinion Delphi Jandhyala method novel consensus method |
Language | English |
License | open-access: http://creativecommons.org/licenses/by-nc-nd/4.0/: This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way. |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-c479t-19264de2e17b587f22b0401f5ec7a678451d4276a7d456ca9ecd2685ff4385693 |
ORCID | 0000-0002-7241-7476 |
OpenAccessLink | https://www.tandfonline.com/doi/abs/10.1080/03007995.2020.1816946 |
PMID | 32866051 |
PageCount | 15 |
ParticipantIDs | informaworld_taylorfrancis_310_1080_03007995_2020_1816946 pubmed_primary_32866051 |
PublicationCentury | 2000 |
PublicationDate | 2020-11-01 |
PublicationDateYYYYMMDD | 2020-11-01 |
PublicationDate_xml | – month: 11 year: 2020 text: 2020-11-01 day: 01 |
PublicationDecade | 2020 |
PublicationPlace | England |
PublicationPlace_xml | – name: England |
PublicationTitle | Current medical research and opinion |
PublicationTitleAlternate | Curr Med Res Opin |
PublicationYear | 2020 |
Publisher | Taylor & Francis |
Publisher_xml | – name: Taylor & Francis |
SSID | ssj0012899 |
Score | 2.553145 |
SecondaryResourceType | review_article |
Snippet | Increasing demand for reliable evidence in patient care and its delivery has necessitated the development of several approaches for generating quality... |
SourceID | pubmed informaworld |
SourceType | Index Database Publisher |
StartPage | 1873 |
SubjectTerms | case studies Consensus Delivery of Health Care Delphi Delphi Technique expert opinion Humans Jandhyala method novel consensus method Research Design |
Title | Delphi, non-RAND modified Delphi, RAND/UCLA appropriateness method and a novel group awareness and consensus methodology for consensus measurement: a systematic literature review |
URI | https://www.tandfonline.com/doi/abs/10.1080/03007995.2020.1816946 https://www.ncbi.nlm.nih.gov/pubmed/32866051 |
Volume | 36 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1bS8MwFA46QXwR784befDRsCZN09a3sTmGuCHiQJ9KkyYozDlcp__LX-hJ0k4d-ORLobm1nJPLd5JzviB0LnOR8iiPiaFaEm5EQJJEhMRwrmA1oUq5KNfBUPRH_Pohqr0JZ5VbpbWhjSeKcHO1Hdy5nNUecS3ol4HlMQPrjkFSQuFzYhWtMQCK1qsveOwvDhKsPeEPEgJi69RBPH81s8RYugQ23aLT20KbFVrEba_ebbSiJztofVCdh--iz64eT5-eLzAY8eSuPezil9fi2QCsxHWOTW2NOjdt7OjDp9DhSje_YX95NAYx4BwaeNdj7GI8cP5hI8RsEZunrL_1ZDavK7h9eAw__ytnsdN4CY1900Pj8YK2GfsomT006l3dd_qkuoWBKB6nJQEIKHihmaaxjJLYMCZh4FMTaRXnsNTxiBacxSKPCwBjKk-1KphIImN4mEQiDfdRA4SgDxEOwTbh2mgw2hSXLEwllCgYiJ2mEpBGE6U_hZ-VbtPC-BtGspDWdKaV4jKruKxSXBMdeE1lU0_TkYUsEWCs0aN_tHqMNuyrDz48QY3yba5PAYWU8sz1M3gObwdfsF7WMg |
linkProvider | Taylor & Francis |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LT8MwDI5gSMAF8WY8c-BItCVN05bbtIEGbDugTRqnqk0TMWmMCTr4X_xCnKQdYxInrnHiVraT-HNiB6HLNBER95OAaKpSwrWokzAUHtGcS9hNqJQ2y7XbE-0Bvx_6w4VcGHOt0mBo7QpF2LXaTG4TjC6vxNXAMOumkBnAOwZNIYXviVW05oeAJsCm60_t-UmCARTuJKFOzJgyi-cvNkslS5e8Tbvr3G6jrcJdxA2n3x20oia7aL1bHIjvoa-WGk-fR1cYUDx5bPRa-OU1G2nwK3FJMa21QbPTwLZ--BQsLrcLHHavR2OQA06AwYcaY5vkgZNPkyJmuhiaNBeuJ--zcoANxGP4-V-UeajxGpj91IfG43ndZuzSZPbR4Pam32yT4hkGInkQ5QR8QMEzxRQNUj8MNGMpzHyqfSWDBPY67tOMs0AkQQbemEwiJTMmQl9r7oW-iLwDVAEhqCOEPQAnXGkFqE3ylHlRCj0yBmKnUQquRhVFi8KPcxu10O6JkdijZT3TQnGxUVxcKK6KDp2m4qmr0xF7LBSA1ujxP7heoI12v9uJO3e9hxO0aUguE_EUVfK3mToDlyRPz63NfQM7p9iR |
linkToPdf | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3JTsMwELWgSBUXxE5ZfeCI1dpxnIRb1VIVaCuEqASnKHFsUamUiqbwX3whYzspUIkTVy-TaMbLG3vmGaHzNBER95OAaKpSwrVokDAUHtGcS9hNqJQ2y7U_EN0hv3n0y2jCWRFWaXxo7Ygi7FptJvc002VEXB3GZcPwmIF3x6AopPA5sYrW_DCyGKnx1F1cJBh_wl0kNIjpUybx_CVmibF0CWzaTaeziTYKtIibzrxbaEVNtlG1X9yH76DPthpPn0cXGJx4ct8ctPHLazbSACtxWWNK68NWr4ktffgUBlxu1zfsHo_GoAacgIB3NcY2xwMnHyZDzDQxddLEW09m87KDPYfH8PO_ahYnjZcg7JseGo8XtM3YZcnsomHn6qHVJcUrDETyIMoJQEDBM8UUDVI_DDRjKUx8qn0lgwS2Ou7TjLNAJEEGYEwmkZIZE6GvNfdCX0TeHqqAEtQBwh74JlxpBU6b5CnzohRaZAzUTqMUkEYNRT-VH-f20EK7F0Zij5Z0poXhYmO4uDBcDe07S8VTR9MReywU4KzRw39IPUPVu3Yn7l0Pbo_QuqlxeYjHqJK_zdUJAJI8PbVD7gsAw9e6 |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Delphi%2C+non-RAND+modified+Delphi%2C+RAND%2FUCLA+appropriateness+method+and+a+novel+group+awareness+and+consensus+methodology+for+consensus+measurement%3A+a+systematic+literature+review&rft.jtitle=Current+medical+research+and+opinion&rft.au=Jandhyala%2C+Ravi&rft.date=2020-11-01&rft.pub=Taylor+%26+Francis&rft.issn=0300-7995&rft.eissn=1473-4877&rft.volume=36&rft.issue=11&rft.spage=1873&rft.epage=1887&rft_id=info:doi/10.1080%2F03007995.2020.1816946&rft.externalDBID=0YH&rft.externalDocID=1816946 |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0300-7995&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0300-7995&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0300-7995&client=summon |