Adapted large language models can outperform medical experts in clinical text summarization

Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range o...

Full description

Saved in:
Bibliographic Details
Published inNature medicine Vol. 30; no. 4; pp. 1134 - 1142
Main Authors Van Veen, Dave, Van Uden, Cara, Blankemeier, Louis, Delbrouck, Jean-Benoit, Aali, Asad, Bluethgen, Christian, Pareek, Anuj, Polacin, Malgorzata, Reis, Eduardo Pontes, Seehofnerová, Anna, Rohatgi, Nidhi, Hosamani, Poonam, Collins, William, Ahuja, Neera, Langlotz, Curtis P., Hom, Jason, Gatidis, Sergios, Pauly, John, Chaudhari, Akshay S.
Format Journal Article
LanguageEnglish
Published New York Nature Publishing Group US 01.04.2024
Nature Publishing Group
Subjects
692/308/575
692/700
706/703/559
Adaptation
Biomedical and Life Sciences
Biomedicine
Cancer Research
Chatbots
Documentation
Electronic Health Records
Electronic medical records
Humans
Infectious Diseases
Language
Large language models
Medical personnel
Metabolic Diseases
Molecular Medicine
Natural Language Processing
Neurosciences
Patients
Performance assessment
Physician-Patient Relations
Radiology
Semantics
Summaries
Online AccessGet full text

Cover

Abstract Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor–patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care. Comparative performance assessment of large language models identified ChatGPT-4 as the best-adapted model across a diverse set of clinical text summarization tasks, and it outperformed 10 medical experts in a reader study.
AbstractList Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor–patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.Comparative performance assessment of large language models identified ChatGPT-4 as the best-adapted model across a diverse set of clinical text summarization tasks, and it outperformed 10 medical experts in a reader study.
Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor-patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.
Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor-patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor-patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.
Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor–patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care. Comparative performance assessment of large language models identified ChatGPT-4 as the best-adapted model across a diverse set of clinical text summarization tasks, and it outperformed 10 medical experts in a reader study.
Author Ahuja, Neera
Hom, Jason
Langlotz, Curtis P.
Pareek, Anuj
Hosamani, Poonam
Bluethgen, Christian
Van Veen, Dave
Pauly, John
Blankemeier, Louis
Gatidis, Sergios
Collins, William
Chaudhari, Akshay S.
Polacin, Malgorzata
Seehofnerová, Anna
Rohatgi, Nidhi
Van Uden, Cara
Delbrouck, Jean-Benoit
Reis, Eduardo Pontes
Aali, Asad
AuthorAffiliation 2 Stanford Center for Artificial Intelligence in Medicine and Imaging, Palo Alto, CA, USA
6 Copenhagen University Hospital, Copenhagen, Denmark
8 Department of Medicine, Stanford University, Stanford, CA, USA
4 Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA
3 Department of Computer Science, Stanford University, Stanford, CA, USA
12 Stanford Cardiovascular Institute, Stanford, CA, USA
7 Albert Einstein Israelite Hospital, São Paulo, Brazil
9 Department of Radiology, Stanford University, Stanford, CA, USA
1 Department of Electrical Engineering, Stanford University, Stanford, CA, USA
11 Department of Biomedical Data Science, Stanford University, Stanford, CA, USA
5 Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
10 Department of Neurosurgery, Stanford University, Stanford, CA, USA
AuthorAffiliation_xml – name: 12 Stanford Cardiovascular Institute, Stanford, CA, USA
– name: 10 Department of Neurosurgery, Stanford University, Stanford, CA, USA
– name: 2 Stanford Center for Artificial Intelligence in Medicine and Imaging, Palo Alto, CA, USA
– name: 3 Department of Computer Science, Stanford University, Stanford, CA, USA
– name: 11 Department of Biomedical Data Science, Stanford University, Stanford, CA, USA
– name: 4 Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA
– name: 8 Department of Medicine, Stanford University, Stanford, CA, USA
– name: 5 Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
– name: 9 Department of Radiology, Stanford University, Stanford, CA, USA
– name: 1 Department of Electrical Engineering, Stanford University, Stanford, CA, USA
– name: 6 Copenhagen University Hospital, Copenhagen, Denmark
– name: 7 Albert Einstein Israelite Hospital, São Paulo, Brazil
Author_xml – sequence: 1
  givenname: Dave
  orcidid: 0000-0001-9312-1773
  surname: Van Veen
  fullname: Van Veen, Dave
  email: vanveen@stanford.edu
  organization: Department of Electrical Engineering, Stanford University, Stanford Center for Artificial Intelligence in Medicine and Imaging
– sequence: 2
  givenname: Cara
  surname: Van Uden
  fullname: Van Uden, Cara
  organization: Stanford Center for Artificial Intelligence in Medicine and Imaging, Department of Computer Science, Stanford University
– sequence: 3
  givenname: Louis
  surname: Blankemeier
  fullname: Blankemeier, Louis
  organization: Department of Electrical Engineering, Stanford University, Stanford Center for Artificial Intelligence in Medicine and Imaging
– sequence: 4
  givenname: Jean-Benoit
  surname: Delbrouck
  fullname: Delbrouck, Jean-Benoit
  organization: Stanford Center for Artificial Intelligence in Medicine and Imaging
– sequence: 5
  givenname: Asad
  surname: Aali
  fullname: Aali, Asad
  organization: Department of Electrical and Computer Engineering, The University of Texas at Austin
– sequence: 6
  givenname: Christian
  orcidid: 0000-0001-7321-5676
  surname: Bluethgen
  fullname: Bluethgen, Christian
  organization: Stanford Center for Artificial Intelligence in Medicine and Imaging, Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich
– sequence: 7
  givenname: Anuj
  orcidid: 0000-0002-1526-3685
  surname: Pareek
  fullname: Pareek, Anuj
  organization: Stanford Center for Artificial Intelligence in Medicine and Imaging, Copenhagen University Hospital
– sequence: 8
  givenname: Malgorzata
  surname: Polacin
  fullname: Polacin, Malgorzata
  organization: Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich
– sequence: 9
  givenname: Eduardo Pontes
  surname: Reis
  fullname: Reis, Eduardo Pontes
  organization: Stanford Center for Artificial Intelligence in Medicine and Imaging, Albert Einstein Israelite Hospital
– sequence: 10
  givenname: Anna
  surname: Seehofnerová
  fullname: Seehofnerová, Anna
  organization: Department of Medicine, Stanford University, Department of Radiology, Stanford University
– sequence: 11
  givenname: Nidhi
  orcidid: 0000-0003-4574-0283
  surname: Rohatgi
  fullname: Rohatgi, Nidhi
  organization: Department of Medicine, Stanford University, Department of Neurosurgery, Stanford University
– sequence: 12
  givenname: Poonam
  surname: Hosamani
  fullname: Hosamani, Poonam
  organization: Department of Medicine, Stanford University
– sequence: 13
  givenname: William
  orcidid: 0000-0003-0974-2599
  surname: Collins
  fullname: Collins, William
  organization: Department of Medicine, Stanford University
– sequence: 14
  givenname: Neera
  surname: Ahuja
  fullname: Ahuja, Neera
  organization: Department of Medicine, Stanford University
– sequence: 15
  givenname: Curtis P.
  orcidid: 0000-0002-8972-8051
  surname: Langlotz
  fullname: Langlotz, Curtis P.
  organization: Stanford Center for Artificial Intelligence in Medicine and Imaging, Department of Medicine, Stanford University, Department of Radiology, Stanford University, Department of Biomedical Data Science, Stanford University
– sequence: 16
  givenname: Jason
  surname: Hom
  fullname: Hom, Jason
  organization: Department of Medicine, Stanford University
– sequence: 17
  givenname: Sergios
  surname: Gatidis
  fullname: Gatidis, Sergios
  organization: Stanford Center for Artificial Intelligence in Medicine and Imaging, Department of Radiology, Stanford University
– sequence: 18
  givenname: John
  surname: Pauly
  fullname: Pauly, John
  organization: Department of Electrical Engineering, Stanford University
– sequence: 19
  givenname: Akshay S.
  orcidid: 0000-0002-3667-6796
  surname: Chaudhari
  fullname: Chaudhari, Akshay S.
  organization: Stanford Center for Artificial Intelligence in Medicine and Imaging, Department of Radiology, Stanford University, Department of Biomedical Data Science, Stanford University, Stanford Cardiovascular Institute
BackLink https://www.ncbi.nlm.nih.gov/pubmed/38413730$$D View this record in MEDLINE/PubMed
BookMark eNp9Uctu1TAQtVARfcAPsECR2HQTGMfvFaqq8pAqsWgXSCwsJ5lcXCX2xXZQ6dfX7S3lsUDy2KOZM8fHPodkL8SAhLyk8IYC028zp8LQFjpeQwvRiifkgAouW6rgy17NQelWGyH3yWHOVwDAQJhnZJ9pTplicEC-noxuW3BsZpc2WPewWV1NljjinJvBhSauZYtpimlpFhz94OYGr2ul5MaHZph9uK8VvC5NXpfFJX_jio_hOXk6uTnji4fziFy8P7s8_dief_7w6fTkvB24EqU1XGFH-46C5JT3TjOJ46gouEnR3lAtgXW66ydTY-qx10w5gb1kqDvOjsi7Het27au-AUNJbrbb5KuSnzY6b__uBP_NbuIPSylXRgpTGY4fGFL8vmIudvF5wLl-BsY1286wugxwqNDX_0Cv4ppCfZ1lwKk0EuBOUrdDDSnmnHB6VEPB3llnd9bZap29t86KOvTqz3c8jvzyqgLYDpBrK2ww_b77P7S3xu-nnw
CitedBy_id crossref_primary_10_1038_s41591_024_03097_1
crossref_primary_10_1002_adma_202405163
crossref_primary_10_1200_EDBK_438516
crossref_primary_10_3389_fpubh_2024_1352979
crossref_primary_10_3390_app14072817
crossref_primary_10_3389_frdem_2024_1385303
crossref_primary_10_1001_jamasurg_2024_1621
crossref_primary_10_2196_53289
crossref_primary_10_1055_a_2300_6235
crossref_primary_10_1111_liv_15974
crossref_primary_10_1038_s41591_024_02888_w
Cites_doi 10.1093/jamia/ocy088
10.1097/ACO.0000000000000588
10.7326/M16-0961
10.1038/s41746-023-00879-8
10.1016/j.ebiom.2023.104770
10.1370/afm.2121
10.1097/NAQ.0b013e3181c95ec4
10.1038/s41598-023-43436-9
10.1038/s41591-023-02448-8
10.1016/j.patter.2023.100802
10.1197/jamia.M3294
10.1093/jamia/ocv080
10.1097/CIN.0000000000000222
10.1038/s41746-023-00939-z
10.1016/S2589-7500(23)00225-X
10.1016/j.jcm.2016.02.012
10.1097/MD.0000000000012319
10.1097/TA.0000000000000986
10.1016/j.mayocp.2016.05.007
10.21105/joss.01026
10.3390/ijerph10062214
10.1097/MLR.0000000000000679
10.1145/3419106
10.1038/s41597-019-0322-0
10.48550/arXiv.2306.05685
10.48550/arXiv.2304.07437
10.1038/s41746-023-00896-7
10.3115/1073083.1073135
10.48550/arXiv.2304.14670
10.1038/s41597-023-02487-3
10.48550/arXiv.2308.14089
10.48550/arXiv.2305.14314
10.48550/arXiv.2307.14334
10.18653/v1/2020.emnlp-demos.6
10.48550/arXiv.2304.08247
10.48550/arXiv.2205.05131
10.18653/v1/2023.clinicalnlp-1.52
10.48550/arXiv.2301.13688
10.48550/arXiv.2306.17384
10.1038/s41586-023-06291-2
10.1101/2023.06.04.23290939
10.13026/1z6g-ex18
10.48550/arXiv.2304.08448
10.48550/arXiv.2006.06292
10.18653/v1/2023.bionlp-1.42
10.1145/3641289
10.1007/978-3-031-20627-6_1
10.48550/arXiv.1602.02410
10.1161/01.CIR.101.23.e215
10.13026/C2HM2Q
10.18653/v1/2022.findings-emnlp.38
10.48550/arXiv.2303.12712
10.48550/arXiv.2307.02486
10.48550/arXiv.2210.17323
10.18653/v1/2023.bionlp-1.51
10.48550/arXiv.2303.08774
10.18653/v1/2023.acl-short.41
10.48550/arXiv.2307.09288
10.18653/v1/2023.bionlp-1.43
10.18653/v1/P19-1215
10.18653/v1/P18-1008
10.48550/arXiv.2106.09685
10.18653/v1/2023.bionlp-1.45
10.48550/arXiv.2303.18223
10.48550/arXiv.2210.11416
10.48550/arXiv.2212.02216
10.48550/arXiv.2305.12031
ContentType Journal Article
Copyright The Author(s), under exclusive licence to Springer Nature America, Inc. 2024. corrected publication 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
2024. The Author(s), under exclusive licence to Springer Nature America, Inc.
Copyright_xml – notice: The Author(s), under exclusive licence to Springer Nature America, Inc. 2024. corrected publication 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
– notice: 2024. The Author(s), under exclusive licence to Springer Nature America, Inc.
DBID CGR
CUY
CVF
ECM
EIF
NPM
AAYXX
CITATION
7QG
7QL
7QP
7QR
7T5
7TK
7TM
7TO
7U7
7U9
8FD
C1K
FR3
H94
K9.
M7N
P64
RC3
7X8
5PM
DOI 10.1038/s41591-024-02855-5
DatabaseName Medline
MEDLINE
MEDLINE (Ovid)
MEDLINE
MEDLINE
PubMed
CrossRef
Animal Behavior Abstracts
Bacteriology Abstracts (Microbiology B)
Calcium & Calcified Tissue Abstracts
Chemoreception Abstracts
Immunology Abstracts
Neurosciences Abstracts
Nucleic Acids Abstracts
Oncogenes and Growth Factors Abstracts
Toxicology Abstracts
Virology and AIDS Abstracts
Technology Research Database
Environmental Sciences and Pollution Management
Engineering Research Database
AIDS and Cancer Research Abstracts
ProQuest Health & Medical Complete (Alumni)
Algology Mycology and Protozoology Abstracts (Microbiology C)
Biotechnology and BioEngineering Abstracts
Genetics Abstracts
MEDLINE - Academic
PubMed Central (Full Participant titles)
DatabaseTitle MEDLINE
Medline Complete
MEDLINE with Full Text
PubMed
MEDLINE (Ovid)
CrossRef
Virology and AIDS Abstracts
Oncogenes and Growth Factors Abstracts
Technology Research Database
Toxicology Abstracts
Nucleic Acids Abstracts
ProQuest Health & Medical Complete (Alumni)
Neurosciences Abstracts
Biotechnology and BioEngineering Abstracts
Environmental Sciences and Pollution Management
Genetics Abstracts
Animal Behavior Abstracts
Bacteriology Abstracts (Microbiology B)
Algology Mycology and Protozoology Abstracts (Microbiology C)
AIDS and Cancer Research Abstracts
Chemoreception Abstracts
Immunology Abstracts
Engineering Research Database
Calcium & Calcified Tissue Abstracts
MEDLINE - Academic
DatabaseTitleList Virology and AIDS Abstracts
MEDLINE

MEDLINE - Academic
Database_xml – sequence: 1
  dbid: NPM
  name: PubMed
  url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
  sourceTypes: Index Database
– sequence: 2
  dbid: EIF
  name: MEDLINE
  url: https://proxy.k.utb.cz/login?url=https://www.webofscience.com/wos/medline/basic-search
  sourceTypes: Index Database
DeliveryMethod fulltext_linktorsrc
Discipline Medicine
Biology
EISSN 1546-170X
EndPage 1142
ExternalDocumentID 10_1038_s41591_024_02855_5
38413730
Genre Journal Article
GrantInformation_xml – fundername: Foundation for the National Institutes of Health (Foundation for the National Institutes of Health, Inc.)
  grantid: R01 AR077604; R01 EB002524; R01 AR079431; P41 EB027060
  funderid: https://doi.org/10.13039/100000009
– fundername: NIAMS NIH HHS
  grantid: R01 AR077604
– fundername: NIAMS NIH HHS
  grantid: R01 AR079431
– fundername: NHLBI NIH HHS
  grantid: R01 HL169345
– fundername: NIBIB NIH HHS
  grantid: R01 EB002524
– fundername: NHLBI NIH HHS
  grantid: R01 HL167974
– fundername: NIBIB NIH HHS
  grantid: P41 EB027060
GroupedDBID ---
.-4
.55
.GJ
0R~
123
1CY
29M
2FS
36B
39C
3O-
3V.
4.4
53G
5BI
5M7
5RE
5S5
70F
7X7
85S
88A
88E
88I
8AO
8FE
8FH
8FI
8FJ
8G5
8R4
8R5
AAEEF
AARCD
AAYOK
AAZLF
ABAWZ
ABCQX
ABDBF
ABEFU
ABJNI
ABLJU
ABOCM
ABUWG
ABVXF
ACGFO
ACGFS
ACGOD
ACIWK
ACMJI
ACPRK
ADBBV
ADFRT
AENEX
AFBBN
AFKRA
AFRAH
AFSHS
AGAYW
AGCDD
AGEZK
AGHTU
AHBCP
AHMBA
AHOSX
AHSBF
AIBTJ
ALFFA
ALIPV
ALMA_UNASSIGNED_HOLDINGS
AMTXH
ARMCB
ASPBG
AVWKF
AXYYD
AZFZN
AZQEC
B0M
BBNVY
BENPR
BHPHI
BKKNO
BPHCQ
BVXVI
CCPQU
CS3
DB5
DU5
DWQXO
EAD
EAP
EBC
EBD
EBS
EE.
EJD
EMB
EMK
EMOBN
EPL
ESX
EXGXG
F5P
FEDTE
FQGFK
FSGXE
FYUFA
G8K
GNUQQ
GUQSH
GX1
HCIFZ
HMCUK
HVGLF
HZ~
IAO
IEA
IH2
IHR
IHW
INH
INR
IOF
IOV
ISR
ITC
J5H
L7B
LGEZI
LK8
LOTEE
M0L
M1P
M2O
M2P
M7P
MK0
N9A
NADUK
NNMJJ
NXXTH
O9-
ODYON
P2P
PQQKQ
PROAC
PSQYO
Q2X
RIG
RNS
RNT
RNTTT
RVV
SHXYY
SIXXV
SJN
SNYQT
SV3
TAE
TAOOD
TBHMF
TDRGL
TSG
TUS
UKHRP
UQL
X7M
XJT
YHZ
ZGI
~8M
AAYZH
ABDPE
CGR
CUY
CVF
ECM
EIF
NPM
AAYXX
CITATION
7QG
7QL
7QP
7QR
7T5
7TK
7TM
7TO
7U7
7U9
8FD
ACBWK
C1K
FR3
H94
K9.
M7N
P64
RC3
7X8
5PM
ID FETCH-LOGICAL-c475t-947e21b2106414ba836edd710af71b918603282bf92bffbeb837a5eb63e8243
ISSN 1078-8956
1546-170X
IngestDate Wed Oct 16 06:23:37 EDT 2024
Sat Oct 26 16:49:04 EDT 2024
Tue Nov 12 23:11:28 EST 2024
Thu Sep 12 20:54:52 EDT 2024
Sat Nov 02 12:32:12 EDT 2024
Fri Oct 11 20:44:42 EDT 2024
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 4
Language English
License 2024. The Author(s), under exclusive licence to Springer Nature America, Inc.
LinkModel OpenURL
MergedId FETCHMERGED-LOGICAL-c475t-947e21b2106414ba836edd710af71b918603282bf92bffbeb837a5eb63e8243
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
D.V.V. collected data, developed code, ran experiments, designed reader studies, analyzed results, created figures and wrote the manuscript. All authors reviewed the manuscript and provided meaningful revisions and feedback. C.V.U., L.B. and J.B.D. provided technical advice, in addition to conducting qualitative analysis (C.V.U.), building infrastructure for the Azure API (L.B.) and implementing the MEDCON metric (J.B.). A.A. assisted in model fine-tuning. C.B., A.P., M.P., E.P.R. and A.S. participated in the reader study as radiologists. N.R., P.H., W.C., N.A. and J.H. participated in the reader study as hospitalists. C.P.L., J.P. and A.S.C. provided student funding. S.G. advised on study design, for which J.H. and J.P. provided additional feedback. J.P. and A.S.C. guided the project, with A.S.C. serving as principal investigator and advising on technical details and overall direction. No funders or third parties were involved in study design, analysis or writing.
Author contributions
ORCID 0000-0002-1526-3685
0000-0001-7321-5676
0000-0003-0974-2599
0000-0002-8972-8051
0000-0002-3667-6796
0000-0003-4574-0283
0000-0001-9312-1773
OpenAccessLink https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10635391
PMID 38413730
PQID 3041696004
PQPubID 33975
PageCount 9
ParticipantIDs pubmedcentral_primary_oai_pubmedcentral_nih_gov_11479659
proquest_miscellaneous_2932939040
proquest_journals_3041696004
crossref_primary_10_1038_s41591_024_02855_5
pubmed_primary_38413730
springer_journals_10_1038_s41591_024_02855_5
PublicationCentury 2000
PublicationDate 2024-04-01
PublicationDateYYYYMMDD 2024-04-01
PublicationDate_xml – month: 04
  year: 2024
  text: 2024-04-01
  day: 01
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
– name: United States
PublicationTitle Nature medicine
PublicationTitleAbbrev Nat Med
PublicationTitleAlternate Nat Med
PublicationYear 2024
Publisher Nature Publishing Group US
Nature Publishing Group
Publisher_xml – name: Nature Publishing Group US
– name: Nature Publishing Group
References Strobelt (CR45) 2022; 29
Duffy, Kharasch, Du (CR12) 2010; 34
Brin (CR67) 2023; 13
CR39
CR38
CR37
CR36
CR35
CR79
CR34
CR78
CR33
CR77
CR32
CR76
Yackel, Embi (CR4) 2010; 17
Gesner, Gazarian, Dykes (CR7) 2019; 21
CR31
CR75
Arndt (CR2) 2017; 15
CR30
CR74
CR73
CR71
CR70
Golob, Como, Claridge (CR1) 2016; 80
CR3
Chang, Lee, Liu, Mills (CR13) 2016; 34
CR6
Ratwani (CR8) 2018; 25
CR5
Thirunavukarasu (CR23) 2023; 29
CR49
CR48
CR47
CR46
CR89
Ehrenfeld, Wanderer (CR9) 2018; 31
CR44
CR88
CR87
CR42
CR86
CR41
CR85
Robinson, Kersey (CR15) 2018; 97
CR40
CR84
CR83
CR82
CR81
CR80
Demner-Fushman (CR72) 2016; 23
Wornow (CR22) 2023; 6
Vallat (CR91) 2018; 3
CR19
CR18
CR17
CR16
CR57
CR56
CR54
CR53
CR52
CR51
CR50
Omiye, Lester, Spichak, Rotemberg, Daneshjou (CR58) 2023; 6
Raffel (CR62) 2020; 21
Lim (CR65) 2023; 95
Sinsky (CR10) 2016; 165
Koo, Li (CR90) 2016; 15
Yu (CR55) 2023; 4
Shanafelt (CR14) 2016; 91
CR29
CR28
CR27
CR26
CR25
CR69
Khamisa, Peltzer, Oldenburg (CR11) 2013; 10
CR24
CR68
Walsh (CR43) 2017; 55
CR66
CR21
CR20
CR64
Zack (CR59) 2024; 6
CR63
CR61
CR60
37961377 - Res Sq. 2023 Oct 30:rs.3.rs-3483777. doi: 10.21203/rs.3.rs-3483777/v1
JF Golob Jr (2855_CR1) 2016; 80
2855_CR19
F Yu (2855_CR55) 2023; 4
JM Ehrenfeld (2855_CR9) 2018; 31
RM Ratwani (2855_CR8) 2018; 25
N Khamisa (2855_CR11) 2013; 10
2855_CR52
D Demner-Fushman (2855_CR72) 2016; 23
2855_CR6
2855_CR51
2855_CR5
2855_CR54
2855_CR53
2855_CR3
2855_CR50
D Brin (2855_CR67) 2023; 13
2855_CR16
2855_CR18
2855_CR17
2855_CR56
KE Walsh (2855_CR43) 2017; 55
E Gesner (2855_CR7) 2019; 21
2855_CR57
ZW Lim (2855_CR65) 2023; 95
T Zack (2855_CR59) 2024; 6
2855_CR63
2855_CR21
2855_CR20
2855_CR64
BG Arndt (2855_CR2) 2017; 15
M Wornow (2855_CR22) 2023; 6
2855_CR61
2855_CR60
2855_CR27
2855_CR26
2855_CR29
C Sinsky (2855_CR10) 2016; 165
AJ Thirunavukarasu (2855_CR23) 2023; 29
2855_CR28
TR Yackel (2855_CR4) 2010; 17
2855_CR66
2855_CR25
2855_CR69
2855_CR24
2855_CR68
2855_CR30
2855_CR74
2855_CR73
2855_CR32
2855_CR76
2855_CR31
2855_CR75
2855_CR70
2855_CR71
2855_CR38
KE Robinson (2855_CR15) 2018; 97
2855_CR37
H Strobelt (2855_CR45) 2022; 29
2855_CR39
2855_CR34
2855_CR78
2855_CR33
2855_CR77
2855_CR36
WJ Duffy (2855_CR12) 2010; 34
2855_CR35
2855_CR79
C-P Chang (2855_CR13) 2016; 34
C Raffel (2855_CR62) 2020; 21
JA Omiye (2855_CR58) 2023; 6
TK Koo (2855_CR90) 2016; 15
TD Shanafelt (2855_CR14) 2016; 91
2855_CR41
2855_CR85
2855_CR40
2855_CR84
2855_CR87
2855_CR42
2855_CR86
2855_CR81
2855_CR80
2855_CR83
2855_CR82
2855_CR49
2855_CR48
2855_CR89
2855_CR44
2855_CR88
2855_CR47
R Vallat (2855_CR91) 2018; 3
2855_CR46
References_xml – ident: CR70
– ident: CR49
– ident: CR68
– ident: CR74
– volume: 25
  start-page: 1197
  year: 2018
  end-page: 1201
  ident: CR8
  article-title: A usability and safety analysis of electronic health records: a multi-center study
  publication-title: J. Am. Med. Inform. Assoc.
  doi: 10.1093/jamia/ocy088
  contributor:
    fullname: Ratwani
– ident: CR39
– ident: CR87
– ident: CR16
– volume: 31
  start-page: 357
  year: 2018
  end-page: 360
  ident: CR9
  article-title: Technology as friend or foe? Do electronic health records increase burnout?
  publication-title: Curr. Opin. Anaesthesiol.
  doi: 10.1097/ACO.0000000000000588
  contributor:
    fullname: Wanderer
– ident: CR51
– ident: CR35
– volume: 29
  start-page: 1146
  year: 2022
  end-page: 1156
  ident: CR45
  article-title: Interactive and visual prompt engineering for ad-hoc task adaptation with large language models
  publication-title: IEEE Trans. Vis. Comput. Graph.
  contributor:
    fullname: Strobelt
– ident: CR29
– ident: CR54
– ident: CR61
– ident: CR80
– volume: 165
  start-page: 753
  year: 2016
  end-page: 760
  ident: CR10
  article-title: Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties
  publication-title: Ann. Intern. Med.
  doi: 10.7326/M16-0961
  contributor:
    fullname: Sinsky
– ident: CR77
– ident: CR84
– ident: CR25
– ident: CR42
– volume: 6
  year: 2023
  ident: CR22
  article-title: The shaky foundations of large language models and foundation models for electronic health records
  publication-title: NPJ Digit. Med.
  doi: 10.1038/s41746-023-00879-8
  contributor:
    fullname: Wornow
– ident: CR21
– ident: CR46
– ident: CR71
– ident: CR19
– volume: 95
  start-page: 104770
  year: 2023
  ident: CR65
  article-title: Benchmarking large language models’ performances for myopia care: a comparative analysis of ChatGPT-3.5, ChatGPT-4.0, and Google Bard
  publication-title: EBioMedicine
  doi: 10.1016/j.ebiom.2023.104770
  contributor:
    fullname: Lim
– volume: 15
  start-page: 419
  year: 2017
  end-page: 426
  ident: CR2
  article-title: Tethered to the EHR: primary care physician workload assessment using EHR event log data and time–motion observations
  publication-title: Ann. Fam. Med.
  doi: 10.1370/afm.2121
  contributor:
    fullname: Arndt
– volume: 34
  start-page: E1
  year: 2010
  end-page: E10
  ident: CR12
  article-title: Point of care documentation impact on the nurse–patient interaction
  publication-title: Nurs. Adm. Q.
  doi: 10.1097/NAQ.0b013e3181c95ec4
  contributor:
    fullname: Du
– ident: CR75
– ident: CR88
– ident: CR50
– volume: 13
  year: 2023
  ident: CR67
  article-title: Comparing ChatGPT and GPT-4 performance in USMLE soft skill assessments
  publication-title: Sci. Rep.
  doi: 10.1038/s41598-023-43436-9
  contributor:
    fullname: Brin
– volume: 29
  start-page: 1930
  year: 2023
  end-page: 1940
  ident: CR23
  article-title: Large language models in medicine
  publication-title: Nat. Med.
  doi: 10.1038/s41591-023-02448-8
  contributor:
    fullname: Thirunavukarasu
– ident: CR57
– ident: CR32
– ident: CR60
– ident: CR36
– ident: CR78
– ident: CR85
– ident: CR5
– ident: CR81
– ident: CR64
– ident: CR26
– volume: 4
  start-page: 100802
  year: 2023
  ident: CR55
  article-title: Evaluating progress in automatic chest x-ray radiology report generation
  publication-title: Patterns (N Y)
  doi: 10.1016/j.patter.2023.100802
  contributor:
    fullname: Yu
– volume: 17
  start-page: 104
  year: 2010
  end-page: 107
  ident: CR4
  article-title: Unintended errors with EHR-based result management: a case series
  publication-title: J. Am. Med. Inform. Assoc.
  doi: 10.1197/jamia.M3294
  contributor:
    fullname: Embi
– volume: 23
  start-page: 304
  year: 2016
  end-page: 310
  ident: CR72
  article-title: Preparing a collection of radiology examinations for distribution and retrieval
  publication-title: J. Am. Med. Inform. Assoc.
  doi: 10.1093/jamia/ocv080
  contributor:
    fullname: Demner-Fushman
– ident: CR18
– ident: CR66
– volume: 34
  start-page: 183
  year: 2016
  end-page: 190
  ident: CR13
  article-title: Nurses’ experiences of an initial and reimplemented electronic health record use
  publication-title: Comput. Inform. Nurs.
  doi: 10.1097/CIN.0000000000000222
  contributor:
    fullname: Mills
– ident: CR47
– ident: CR37
– ident: CR53
– ident: CR89
– ident: CR30
– ident: CR33
– volume: 6
  year: 2023
  ident: CR58
  article-title: Large language models propagate race-based medicine
  publication-title: NPJ Digit. Med.
  doi: 10.1038/s41746-023-00939-z
  contributor:
    fullname: Daneshjou
– ident: CR82
– volume: 6
  start-page: e12
  year: 2024
  end-page: e22
  ident: CR59
  article-title: Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study
  publication-title: Lancet Digit. Health
  doi: 10.1016/S2589-7500(23)00225-X
  contributor:
    fullname: Zack
– ident: CR6
– ident: CR79
– ident: CR56
– ident: CR86
– ident: CR40
– ident: CR63
– ident: CR27
– volume: 21
  start-page: 1194
  year: 2019
  end-page: 1198
  ident: CR7
  article-title: The burden and burnout in documenting patient care: an integrative literature review
  publication-title: Stud. Health Technol. Inform.
  contributor:
    fullname: Dykes
– ident: CR69
– volume: 15
  start-page: 155
  year: 2016
  end-page: 163
  ident: CR90
  article-title: A guideline of selecting and reporting intraclass correlation coefficients for reliability research
  publication-title: J. Chiropr. Med.
  doi: 10.1016/j.jcm.2016.02.012
  contributor:
    fullname: Li
– volume: 97
  start-page: e12319
  year: 2018
  ident: CR15
  article-title: Novel electronic health record (EHR) education intervention in large healthcare organization improves quality, efficiency, time, and impact on burnout
  publication-title: Medicine (Baltimore)
  doi: 10.1097/MD.0000000000012319
  contributor:
    fullname: Kersey
– ident: CR44
– ident: CR48
– ident: CR73
– ident: CR3
– ident: CR38
– ident: CR52
– ident: CR17
– volume: 80
  start-page: 742
  year: 2016
  end-page: 747
  ident: CR1
  article-title: The painful truth: the documentation burden of a trauma surgeon
  publication-title: J. Trauma Acute Care Surg.
  doi: 10.1097/TA.0000000000000986
  contributor:
    fullname: Claridge
– ident: CR31
– volume: 21
  start-page: 5485
  year: 2020
  end-page: 5551
  ident: CR62
  article-title: Exploring the limits of transfer learning with a unified text-to-text transformer
  publication-title: J. Mach. Learn. Res.
  contributor:
    fullname: Raffel
– volume: 91
  start-page: 836
  year: 2016
  end-page: 848
  ident: CR14
  article-title: Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction.
  publication-title: Mayo Clin. Proc.
  doi: 10.1016/j.mayocp.2016.05.007
  contributor:
    fullname: Shanafelt
– ident: CR34
– volume: 3
  start-page: 1026
  year: 2018
  ident: CR91
  article-title: Pingouin: statistics in Python
  publication-title: J. Open Source Softw.
  doi: 10.21105/joss.01026
  contributor:
    fullname: Vallat
– ident: CR76
– ident: CR83
– ident: CR28
– ident: CR41
– ident: CR24
– volume: 10
  start-page: 2214
  year: 2013
  end-page: 2240
  ident: CR11
  article-title: Burnout in relation to specific contributing factors and health outcomes among nurses: a systematic review
  publication-title: Int. J. Environ. Res. Public Health
  doi: 10.3390/ijerph10062214
  contributor:
    fullname: Oldenburg
– volume: 55
  start-page: 436–441
  year: 2017
  ident: CR43
  article-title: Measuring harm in healthcare: optimizing adverse event review
  publication-title: Med. Care
  doi: 10.1097/MLR.0000000000000679
  contributor:
    fullname: Walsh
– ident: CR20
– volume: 29
  start-page: 1146
  year: 2022
  ident: 2855_CR45
  publication-title: IEEE Trans. Vis. Comput. Graph.
  contributor:
    fullname: H Strobelt
– ident: 2855_CR61
  doi: 10.1145/3419106
– volume: 3
  start-page: 1026
  year: 2018
  ident: 2855_CR91
  publication-title: J. Open Source Softw.
  doi: 10.21105/joss.01026
  contributor:
    fullname: R Vallat
– ident: 2855_CR73
  doi: 10.1038/s41597-019-0322-0
– volume: 4
  start-page: 100802
  year: 2023
  ident: 2855_CR55
  publication-title: Patterns (N Y)
  doi: 10.1016/j.patter.2023.100802
  contributor:
    fullname: F Yu
– ident: 2855_CR88
– ident: 2855_CR21
  doi: 10.48550/arXiv.2306.05685
– ident: 2855_CR53
  doi: 10.48550/arXiv.2304.07437
– volume: 165
  start-page: 753
  year: 2016
  ident: 2855_CR10
  publication-title: Ann. Intern. Med.
  doi: 10.7326/M16-0961
  contributor:
    fullname: C Sinsky
– ident: 2855_CR56
  doi: 10.1038/s41746-023-00896-7
– ident: 2855_CR71
– ident: 2855_CR42
  doi: 10.3115/1073083.1073135
– ident: 2855_CR46
  doi: 10.48550/arXiv.2304.14670
– ident: 2855_CR17
– ident: 2855_CR75
– ident: 2855_CR41
  doi: 10.1038/s41597-023-02487-3
– ident: 2855_CR3
  doi: 10.48550/arXiv.2308.14089
– ident: 2855_CR40
  doi: 10.48550/arXiv.2305.14314
– volume: 23
  start-page: 304
  year: 2016
  ident: 2855_CR72
  publication-title: J. Am. Med. Inform. Assoc.
  doi: 10.1093/jamia/ocv080
  contributor:
    fullname: D Demner-Fushman
– ident: 2855_CR84
– ident: 2855_CR25
  doi: 10.48550/arXiv.2307.14334
– ident: 2855_CR87
  doi: 10.18653/v1/2020.emnlp-demos.6
– volume: 95
  start-page: 104770
  year: 2023
  ident: 2855_CR65
  publication-title: EBioMedicine
  doi: 10.1016/j.ebiom.2023.104770
  contributor:
    fullname: ZW Lim
– ident: 2855_CR34
  doi: 10.48550/arXiv.2304.08247
– volume: 15
  start-page: 419
  year: 2017
  ident: 2855_CR2
  publication-title: Ann. Fam. Med.
  doi: 10.1370/afm.2121
  contributor:
    fullname: BG Arndt
– ident: 2855_CR32
  doi: 10.48550/arXiv.2205.05131
– volume: 6
  year: 2023
  ident: 2855_CR58
  publication-title: NPJ Digit. Med.
  doi: 10.1038/s41746-023-00939-z
  contributor:
    fullname: JA Omiye
– volume: 34
  start-page: E1
  year: 2010
  ident: 2855_CR12
  publication-title: Nurs. Adm. Q.
  doi: 10.1097/NAQ.0b013e3181c95ec4
  contributor:
    fullname: WJ Duffy
– volume: 6
  start-page: e12
  year: 2024
  ident: 2855_CR59
  publication-title: Lancet Digit. Health
  doi: 10.1016/S2589-7500(23)00225-X
  contributor:
    fullname: T Zack
– ident: 2855_CR35
– ident: 2855_CR82
  doi: 10.18653/v1/2023.clinicalnlp-1.52
– ident: 2855_CR63
  doi: 10.48550/arXiv.2301.13688
– volume: 34
  start-page: 183
  year: 2016
  ident: 2855_CR13
  publication-title: Comput. Inform. Nurs.
  doi: 10.1097/CIN.0000000000000222
  contributor:
    fullname: C-P Chang
– volume: 21
  start-page: 5485
  year: 2020
  ident: 2855_CR62
  publication-title: J. Mach. Learn. Res.
  contributor:
    fullname: C Raffel
– ident: 2855_CR28
  doi: 10.48550/arXiv.2306.17384
– ident: 2855_CR6
– ident: 2855_CR51
– ident: 2855_CR24
  doi: 10.1038/s41586-023-06291-2
– volume: 97
  start-page: e12319
  year: 2018
  ident: 2855_CR15
  publication-title: Medicine (Baltimore)
  doi: 10.1097/MD.0000000000012319
  contributor:
    fullname: KE Robinson
– volume: 31
  start-page: 357
  year: 2018
  ident: 2855_CR9
  publication-title: Curr. Opin. Anaesthesiol.
  doi: 10.1097/ACO.0000000000000588
  contributor:
    fullname: JM Ehrenfeld
– ident: 2855_CR76
– volume: 21
  start-page: 1194
  year: 2019
  ident: 2855_CR7
  publication-title: Stud. Health Technol. Inform.
  contributor:
    fullname: E Gesner
– volume: 91
  start-page: 836
  year: 2016
  ident: 2855_CR14
  publication-title: Mayo Clin. Proc.
  doi: 10.1016/j.mayocp.2016.05.007
  contributor:
    fullname: TD Shanafelt
– ident: 2855_CR66
  doi: 10.1101/2023.06.04.23290939
– ident: 2855_CR49
– ident: 2855_CR20
– ident: 2855_CR80
  doi: 10.13026/1z6g-ex18
– ident: 2855_CR52
  doi: 10.48550/arXiv.2304.08448
– ident: 2855_CR16
  doi: 10.48550/arXiv.2006.06292
– ident: 2855_CR27
  doi: 10.18653/v1/2023.bionlp-1.42
– ident: 2855_CR48
  doi: 10.1145/3641289
– ident: 2855_CR68
  doi: 10.1007/978-3-031-20627-6_1
– ident: 2855_CR47
  doi: 10.48550/arXiv.1602.02410
– ident: 2855_CR81
  doi: 10.1161/01.CIR.101.23.e215
– ident: 2855_CR86
– ident: 2855_CR57
  doi: 10.13026/C2HM2Q
– volume: 25
  start-page: 1197
  year: 2018
  ident: 2855_CR8
  publication-title: J. Am. Med. Inform. Assoc.
  doi: 10.1093/jamia/ocy088
  contributor:
    fullname: RM Ratwani
– volume: 55
  start-page: 436–441
  year: 2017
  ident: 2855_CR43
  publication-title: Med. Care
  doi: 10.1097/MLR.0000000000000679
  contributor:
    fullname: KE Walsh
– volume: 15
  start-page: 155
  year: 2016
  ident: 2855_CR90
  publication-title: J. Chiropr. Med.
  doi: 10.1016/j.jcm.2016.02.012
  contributor:
    fullname: TK Koo
– ident: 2855_CR39
  doi: 10.18653/v1/2022.findings-emnlp.38
– ident: 2855_CR19
  doi: 10.48550/arXiv.2303.12712
– ident: 2855_CR50
  doi: 10.48550/arXiv.2307.02486
– ident: 2855_CR85
  doi: 10.48550/arXiv.2210.17323
– ident: 2855_CR29
– ident: 2855_CR30
– ident: 2855_CR54
  doi: 10.18653/v1/2023.bionlp-1.51
– ident: 2855_CR38
  doi: 10.48550/arXiv.2303.08774
– ident: 2855_CR78
  doi: 10.18653/v1/2023.acl-short.41
– ident: 2855_CR36
  doi: 10.48550/arXiv.2307.09288
– ident: 2855_CR44
– ident: 2855_CR89
– ident: 2855_CR64
– ident: 2855_CR79
  doi: 10.18653/v1/2023.bionlp-1.43
– ident: 2855_CR77
  doi: 10.18653/v1/P19-1215
– ident: 2855_CR60
  doi: 10.18653/v1/P18-1008
– volume: 10
  start-page: 2214
  year: 2013
  ident: 2855_CR11
  publication-title: Int. J. Environ. Res. Public Health
  doi: 10.3390/ijerph10062214
  contributor:
    fullname: N Khamisa
– ident: 2855_CR70
  doi: 10.48550/arXiv.2106.09685
– volume: 6
  year: 2023
  ident: 2855_CR22
  publication-title: NPJ Digit. Med.
  doi: 10.1038/s41746-023-00879-8
  contributor:
    fullname: M Wornow
– volume: 13
  year: 2023
  ident: 2855_CR67
  publication-title: Sci. Rep.
  doi: 10.1038/s41598-023-43436-9
  contributor:
    fullname: D Brin
– ident: 2855_CR37
– volume: 80
  start-page: 742
  year: 2016
  ident: 2855_CR1
  publication-title: J. Trauma Acute Care Surg.
  doi: 10.1097/TA.0000000000000986
  contributor:
    fullname: JF Golob Jr
– ident: 2855_CR33
– ident: 2855_CR74
  doi: 10.18653/v1/2023.bionlp-1.45
– ident: 2855_CR5
– volume: 17
  start-page: 104
  year: 2010
  ident: 2855_CR4
  publication-title: J. Am. Med. Inform. Assoc.
  doi: 10.1197/jamia.M3294
  contributor:
    fullname: TR Yackel
– volume: 29
  start-page: 1930
  year: 2023
  ident: 2855_CR23
  publication-title: Nat. Med.
  doi: 10.1038/s41591-023-02448-8
  contributor:
    fullname: AJ Thirunavukarasu
– ident: 2855_CR18
  doi: 10.48550/arXiv.2303.18223
– ident: 2855_CR83
  doi: 10.18653/v1/2023.clinicalnlp-1.52
– ident: 2855_CR31
  doi: 10.48550/arXiv.2210.11416
– ident: 2855_CR69
  doi: 10.48550/arXiv.2212.02216
– ident: 2855_CR26
  doi: 10.48550/arXiv.2305.12031
SSID ssj0003059
Score 2.6982887
Snippet Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time....
SourceID pubmedcentral
proquest
crossref
pubmed
springer
SourceType Open Access Repository
Aggregation Database
Index Database
Publisher
StartPage 1134
SubjectTerms 692/308/575
692/700
706/703/559
Adaptation
Biomedical and Life Sciences
Biomedicine
Cancer Research
Chatbots
Documentation
Electronic Health Records
Electronic medical records
Humans
Infectious Diseases
Language
Large language models
Medical personnel
Metabolic Diseases
Molecular Medicine
Natural Language Processing
Neurosciences
Patients
Performance assessment
Physician-Patient Relations
Radiology
Semantics
Summaries
Title Adapted large language models can outperform medical experts in clinical text summarization
URI https://link.springer.com/article/10.1038/s41591-024-02855-5
https://www.ncbi.nlm.nih.gov/pubmed/38413730
https://www.proquest.com/docview/3041696004
https://www.proquest.com/docview/2932939040
https://pubmed.ncbi.nlm.nih.gov/PMC11479659
Volume 30
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1bb9MwFLbKJqa9IBi3wEBG4m0EmsROnMe2YpomWiGxTZV4iOzG0SK2pGqTB_gN_GiOL7mUgAST2qhNotjJ9-X42D7nM0JvuZ-CU-2tXGitMpdwOnZZsFJhOxmNM0Ik04o380V4dknOl3Q5Gv3sRS3VlXi_-vHHvJK7oAr7AFeVJfsfyLYXhR3wG_CFLSAM23_CeJLytfIYb1Q4dzv0aFa3UaLT4ArW1dqkBphZdABEa_pXOgy2TYtU4R8nJo_N5mX2ndaFFv8cTMNfwfWv7Lr2alX6_v5La85mfNP1-KGC3-StzA1LPpV13q1qL2_EpqyNbT6XvHCnsijzqj8o4fdjWYwdBc_DZTG1KtfWtpLQ9aLxsm987aRM3h9Z0JbU8-wgp7R_jQjXwOIbffct-CEqhEvVxGeUurR_MqC2vtUcCBg02pEtdEd8-_N8BmVESl_xHtr3wWyBvdyfnE6ni7ZlB9sYmxhWc3M2CQuq8GFYgUN00JS26_MMOjLDeNzfJuW1r3PxED2wnRQ8MYx7hEayOEL3zbKl34_Qwdwy4TH6aimINQVxQ0FsKIiBgrijILYUxJaCOC9wQ0GsKIh3KPgEfTn9eDE7c-16He6KRLRyYxJJ3xO-B26uRwRnQSjTFFxYnkWeiD0WKvFGX2QxfDMhBQsiTqUIA8l8EjxFe0VZyOcIp5TxOFqlaSh0IqbwA8p5Cs6WICH3uINOmueZrI0oS6KDKQKWGCASACLRQCTUQcfNI0_sy7tNgjH0RKD3PiYOetMeBtOq5st4Ict6m4AnDJ8YmjkHPTMItcU10DqI7WDXnqBk23ePFPm1lm9vaOagdw3MXb3-fhsv7l7SS3TYvaTHaK_a1PIV-NKVeG0p_gs30ceq
link.rule.ids 230,315,783,787,888,27936,27937
linkProvider Library Specific Holdings
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Adapted+large+language+models+can+outperform+medical+experts+in+clinical+text+summarization&rft.jtitle=Nature+medicine&rft.au=Van+Veen%2C+Dave&rft.au=Van+Uden%2C+Cara&rft.au=Blankemeier%2C+Louis&rft.au=Delbrouck%2C+Jean-Benoit&rft.date=2024-04-01&rft.issn=1078-8956&rft.eissn=1546-170X&rft.volume=30&rft.issue=4&rft.spage=1134&rft.epage=1142&rft_id=info:doi/10.1038%2Fs41591-024-02855-5&rft_id=info%3Apmid%2F38413730&rft.externalDBID=PMC11479659
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1078-8956&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1078-8956&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1078-8956&client=summon