Limitations of Disembodied Computer-Generated Voice to Convey Emotion in Multimedia Lessons

The goal of this study was to determine whether learning processes and outcomes are affected by the emotional tone of the computer-generated voice in a narrated slideshow. In a between-subjects design, participants viewed a narrated slideshow on lightning formation involving a computer-generated fem...

Full description

Saved in:
Bibliographic Details
Published inInternational journal of human-computer interaction Vol. 41; no. 10; pp. 5881 - 5893
Main Authors Zhao, Fangzheng, Mayer, Richard E.
Format Journal Article
LanguageEnglish
Published Taylor & Francis 19.05.2025
Subjects
computer-generated voice
emotion
multimedia learning
narrated slideshow
Text-to-speech
Online AccessGet full text
ISSN1044-7318
1532-7590
DOI10.1080/10447318.2024.2371681

Cover

Abstract The goal of this study was to determine whether learning processes and outcomes are affected by the emotional tone of the computer-generated voice in a narrated slideshow. In a between-subjects design, participants viewed a narrated slideshow on lightning formation involving a computer-generated female (Experiment 1) or male voice (Experiment 2) that displayed happy, content, angry, or sad emotion. On subsequent surveys and post-tests, students could recognize positive or negative emotions conveyed by the female voice but not the male voice. The emotional tone of the instructor's voices had minimal impact on students' ratings of felt emotion during learning, ratings of social connection with the instructor, or scores on retention and transfer tests of learning outcome. The findings highlight the limitations of computer-generated voices to convey emotions that trigger affective, social, and cognitive processes when an onscreen instructor is absent, thereby suggesting a boundary condition for the cognitive-affective model of e-learning.
AbstractList The goal of this study was to determine whether learning processes and outcomes are affected by the emotional tone of the computer-generated voice in a narrated slideshow. In a between-subjects design, participants viewed a narrated slideshow on lightning formation involving a computer-generated female (Experiment 1) or male voice (Experiment 2) that displayed happy, content, angry, or sad emotion. On subsequent surveys and post-tests, students could recognize positive or negative emotions conveyed by the female voice but not the male voice. The emotional tone of the instructor's voices had minimal impact on students' ratings of felt emotion during learning, ratings of social connection with the instructor, or scores on retention and transfer tests of learning outcome. The findings highlight the limitations of computer-generated voices to convey emotions that trigger affective, social, and cognitive processes when an onscreen instructor is absent, thereby suggesting a boundary condition for the cognitive-affective model of e-learning.
Author Mayer, Richard E.
Zhao, Fangzheng
Author_xml – sequence: 1
  givenname: Fangzheng
  orcidid: 0000-0002-3575-9039
  surname: Zhao
  fullname: Zhao, Fangzheng
  organization: Department of Psychological and Brain Sciences, University of California
– sequence: 2
  givenname: Richard E.
  orcidid: 0000-0003-4055-6938
  surname: Mayer
  fullname: Mayer, Richard E.
  organization: Department of Psychological and Brain Sciences, University of California
BookMark eNqFkE1OwzAQRi1UJNrCEZBygRT_JE4iNqBSClIQG2CFZDnOWDJK7Mp2Qb0NZ-FkJGrZsIDVjGa-9y3eDE2ss4DQOcELgkt8QXCWFYyUC4pptqCsILwkR2hKckbTIq_wZNiHTDqGTtAshDeMMcU5m6LX2vQmymicDYnTyY0J0DeuNdAmS9dvthF8ugYLXsbh9OKMgiS64WffYZesejeiibFfnw_bLpoeWiOTGkIYCk_RsZZdgLPDnKPn29XT8i6tH9f3y-s6VYzgmJICpC5ayTjXFafQVFAq3OimyduMEswyRXnRECKZrEpZNZxyznRVctoqVgGbo3zfq7wLwYMWG2966XeCYDEqEj-KxKhIHBQN3OUvTh1cRC9N9y99taeN1c738sP5rhVR7jrntZdWmSDY3xXfVOKDVQ
CitedBy_id crossref_primary_10_1080_00220973_2024_2446169
crossref_primary_10_1177_07356331251322469
Cites_doi 10.1016/j.cobeha.2017.10.003
10.1037/a0028616
10.1016/j.learninstruc.2011.10.001
10.4324/9781410605245-7
10.4324/9780203148211
10.1016/j.compedu.2019.103605
10.1037/h0077714
10.18608/hla17.010
10.1037/0022-3514.64.3.431
10.1177/23328584211040058
10.1007/s40593-021-00238-2
10.1007/s40593-021-00282-y
10.1016/S0022-5371(66)80044-0
10.1016/j.chb.2021.106724
10.1080/1358165042000283101
10.1016/j.cedpsych.2004.07.001
10.1007/s10648-020-09587-1
10.1007/s11423-021-10057-w
10.3102/00346543068003350
10.1016/j.learninstruc.2019.05.010
10.1111/jcal.12335
10.1007/s10639-022-11255-6
10.1037/0022-0663.93.1.187
10.1037/0022-0663.91.2.358
10.1080/87567555.2014.935700
10.1037/0022-0663.100.2.380
10.1007/s11423-023-10228-x
10.5334/jime.519
10.1016/j.chb.2020.106554
10.1016/B978-0-12-801856-9.00007-4
10.2190/EC.49.1.a
10.1146/annurev.psych.55.090902.141555
10.1177/0735633118802877
10.1037/0022-0663.93.2.390
10.1016/j.edurev.2018.05.002
10.1037/0022-0663.94.1.156
10.1007/s10648-007-9047-2
10.1016/j.compedu.2019.103756
10.1037/0022-0663.95.2.419
10.1016/j.compedu.2017.07.003
10.1037/0022-0663.91.4.638
10.1080/15391523.2014.888265
10.1145/258549.258797
10.1037/0033-295X.110.1.145
10.1016/j.learninstruc.2018.08.002
10.1037/0022-0663.92.4.724
ContentType Journal Article
Copyright 2024 The Author(s). Published with license by Taylor & Francis Group, LLC. 2024
Copyright_xml – notice: 2024 The Author(s). Published with license by Taylor & Francis Group, LLC. 2024
DBID 0YH
AAYXX
CITATION
DOI 10.1080/10447318.2024.2371681
DatabaseName Taylor & Francis Open Access
CrossRef
DatabaseTitle CrossRef
DatabaseTitleList
Database_xml – sequence: 1
  dbid: 0YH
  name: Taylor & Francis Open Access
  url: https://www.tandfonline.com
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 1532-7590
EndPage 5893
ExternalDocumentID 10_1080_10447318_2024_2371681
2371681
Genre Research Article
GrantInformation_xml – fundername: National Science Foundation
  grantid: 2201020
GroupedDBID -~X
.7F
.DC
.QJ
0BK
0R~
0YH
29J
30N
4.4
53G
5GY
5VS
8VB
AAENE
AAJMT
AALDU
AAMIU
AAPUL
AAQRR
ABCCY
ABDBF
ABFIM
ABHAV
ABIVO
ABJNI
ABLIJ
ABLJU
ABPAQ
ABPEM
ABPPZ
ABTAI
ABXUL
ABXYU
ACGFS
ACGOD
ACHQT
ACTIO
ADCVX
ADGTB
ADYSH
AEISY
AENEX
AEOZL
AEPSL
AEYOC
AFKVX
AFRVT
AGDLA
AGMYJ
AHDZW
AIJEM
AIYEW
AKBVH
AKOOK
AKVCP
ALMA_UNASSIGNED_HOLDINGS
ALQZU
AMPGV
AQRUH
AVBZW
AWYRJ
BLEHA
CCCUG
CE4
CS3
DGEBU
DKSSO
DU5
EAP
EBR
EBS
EBU
EDO
EST
ESX
E~A
E~B
F5P
FEDTE
FPAXQ
GTTXZ
H13
HF~
HVGLF
HZ~
H~P
IPNFZ
J.P
KYCEM
LJTGL
M4Z
NA5
O9-
P2P
PQQKQ
PZZ
QWB
RIG
RNANH
ROSJB
RTWRZ
S-T
SNACF
TBQAZ
TDBHL
TEN
TFL
TFT
TFW
TNC
TTHFI
TUROJ
TWF
UT5
UU3
ZL0
~S~
AAGDL
AAHIA
AAYXX
CITATION
NX~
ZGOLN
ID FETCH-LOGICAL-c310t-17eaf7da366f962eb9e8c0bfbb5d421034c267b11a3a98a9b62663f9862dc39e3
IEDL.DBID 0YH
ISSN 1044-7318
IngestDate Tue Jul 01 04:59:34 EDT 2025
Thu Apr 24 22:58:59 EDT 2025
Sat May 10 04:10:19 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 10
Language English
License open-access: http://creativecommons.org/licenses/by-nc-nd/4.0/: This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way. The terms on which this article has been published allow the posting of the Accepted Manuscript in a repository by the author(s) or with their consent.
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c310t-17eaf7da366f962eb9e8c0bfbb5d421034c267b11a3a98a9b62663f9862dc39e3
ORCID 0000-0003-4055-6938
0000-0002-3575-9039
OpenAccessLink https://www.tandfonline.com/doi/abs/10.1080/10447318.2024.2371681
PageCount 13
ParticipantIDs informaworld_taylorfrancis_310_1080_10447318_2024_2371681
crossref_citationtrail_10_1080_10447318_2024_2371681
crossref_primary_10_1080_10447318_2024_2371681
ProviderPackageCode CITATION
AAYXX
PublicationCentury 2000
PublicationDate 2025-05-19
PublicationDateYYYYMMDD 2025-05-19
PublicationDate_xml – month: 05
  year: 2025
  text: 2025-05-19
  day: 19
PublicationDecade 2020
PublicationTitle International journal of human-computer interaction
PublicationYear 2025
Publisher Taylor & Francis
Publisher_xml – name: Taylor & Francis
References e_1_3_2_28_1
e_1_3_2_20_1
e_1_3_2_41_1
e_1_3_2_22_1
e_1_3_2_43_1
e_1_3_2_24_1
e_1_3_2_26_1
e_1_3_2_47_1
Pekrun R. (e_1_3_2_45_1) 2016
e_1_3_2_16_1
e_1_3_2_39_1
e_1_3_2_9_1
e_1_3_2_18_1
e_1_3_2_7_1
e_1_3_2_54_1
e_1_3_2_10_1
e_1_3_2_33_1
e_1_3_2_12_1
Mayer R. E. (e_1_3_2_31_1) 2021
e_1_3_2_35_1
e_1_3_2_5_1
Graesser A. C. (e_1_3_2_17_1) 2007; 158
e_1_3_2_14_1
e_1_3_2_37_1
e_1_3_2_56_1
e_1_3_2_3_1
Reeves B. (e_1_3_2_49_1) 1996
e_1_3_2_50_1
Ryu J. E. E. H. E. O. N. (e_1_3_2_52_1) 2005; 2
e_1_3_2_27_1
Nass C. (e_1_3_2_44_1) 2005
Bower G. H. (e_1_3_2_4_1) 1992
e_1_3_2_29_1
e_1_3_2_42_1
e_1_3_2_21_1
e_1_3_2_23_1
e_1_3_2_25_1
Pekrun R. (e_1_3_2_46_1) 2017
e_1_3_2_48_1
Wang F. (e_1_3_2_55_1) 2022
e_1_3_2_40_1
e_1_3_2_38_1
e_1_3_2_8_1
e_1_3_2_19_1
e_1_3_2_2_1
e_1_3_2_30_1
e_1_3_2_11_1
e_1_3_2_32_1
e_1_3_2_53_1
e_1_3_2_6_1
e_1_3_2_13_1
e_1_3_2_34_1
e_1_3_2_36_1
e_1_3_2_57_1
e_1_3_2_51_1
Fiorella L. (e_1_3_2_15_1) 2022
References_xml – ident: e_1_3_2_39_1
  doi: 10.1016/j.cobeha.2017.10.003
– ident: e_1_3_2_34_1
  doi: 10.1037/a0028616
– ident: e_1_3_2_13_1
  doi: 10.1016/j.learninstruc.2011.10.001
– ident: e_1_3_2_14_1
  doi: 10.4324/9781410605245-7
– ident: e_1_3_2_47_1
  doi: 10.4324/9780203148211
– volume-title: The media equation
  year: 1996
  ident: e_1_3_2_49_1
– start-page: 120
  volume-title: Handbook of motivation at school
  year: 2016
  ident: e_1_3_2_45_1
– ident: e_1_3_2_11_1
  doi: 10.1016/j.compedu.2019.103605
– ident: e_1_3_2_50_1
  doi: 10.1037/h0077714
– ident: e_1_3_2_12_1
  doi: 10.18608/hla17.010
– ident: e_1_3_2_2_1
  doi: 10.1037/0022-3514.64.3.431
– ident: e_1_3_2_20_1
  doi: 10.1177/23328584211040058
– ident: e_1_3_2_26_1
  doi: 10.1007/s40593-021-00238-2
– ident: e_1_3_2_23_1
  doi: 10.1007/s40593-021-00282-y
– ident: e_1_3_2_16_1
  doi: 10.1016/S0022-5371(66)80044-0
– ident: e_1_3_2_19_1
  doi: 10.1016/j.chb.2021.106724
– ident: e_1_3_2_7_1
  doi: 10.1080/1358165042000283101
– ident: e_1_3_2_3_1
  doi: 10.1016/j.cedpsych.2004.07.001
– ident: e_1_3_2_5_1
  doi: 10.1007/s10648-020-09587-1
– ident: e_1_3_2_25_1
  doi: 10.1007/s11423-021-10057-w
– ident: e_1_3_2_22_1
  doi: 10.3102/00346543068003350
– volume: 2
  start-page: 291
  issue: 4
  year: 2005
  ident: e_1_3_2_52_1
  article-title: The psychometric structure of pedagogical agent persona
  publication-title: Technology Instruction Cognition and Learning,
– ident: e_1_3_2_32_1
  doi: 10.1016/j.learninstruc.2019.05.010
– ident: e_1_3_2_30_1
  doi: 10.1111/jcal.12335
– ident: e_1_3_2_28_1
  doi: 10.1007/s10639-022-11255-6
– ident: e_1_3_2_35_1
  doi: 10.1037/0022-0663.93.1.187
– ident: e_1_3_2_40_1
  doi: 10.1037/0022-0663.91.2.358
– volume-title: Wired for speech: How voice activates and advances the human-computer relationship
  year: 2005
  ident: e_1_3_2_44_1
– ident: e_1_3_2_21_1
  doi: 10.1080/87567555.2014.935700
– ident: e_1_3_2_36_1
  doi: 10.1037/0022-0663.100.2.380
– start-page: 3
  volume-title: The handbook of emotion and memory: Research and theory
  year: 1992
  ident: e_1_3_2_4_1
– ident: e_1_3_2_57_1
  doi: 10.1007/s11423-023-10228-x
– volume: 158
  start-page: 569
  year: 2007
  ident: e_1_3_2_17_1
  article-title: Emotions and learning with auto tutor
  publication-title: Frontiers in Artificial Intelligence and Applications,
– ident: e_1_3_2_18_1
  doi: 10.5334/jime.519
– ident: e_1_3_2_24_1
  doi: 10.1016/j.chb.2020.106554
– ident: e_1_3_2_48_1
  doi: 10.1016/B978-0-12-801856-9.00007-4
– ident: e_1_3_2_54_1
  doi: 10.2190/EC.49.1.a
– ident: e_1_3_2_56_1
  doi: 10.1146/annurev.psych.55.090902.141555
– ident: e_1_3_2_9_1
  doi: 10.1177/0735633118802877
– ident: e_1_3_2_33_1
  doi: 10.1037/0022-0663.93.2.390
– start-page: 251
  volume-title: Handbook of competence and motivation: Theory and application
  year: 2017
  ident: e_1_3_2_46_1
– ident: e_1_3_2_10_1
  doi: 10.1016/j.edurev.2018.05.002
– ident: e_1_3_2_42_1
  doi: 10.1037/0022-0663.94.1.156
– volume-title: Multimedia learning
  year: 2021
  ident: e_1_3_2_31_1
– ident: e_1_3_2_43_1
  doi: 10.1007/s10648-007-9047-2
– ident: e_1_3_2_6_1
  doi: 10.1016/j.compedu.2019.103756
– ident: e_1_3_2_38_1
  doi: 10.1037/0022-0663.95.2.419
– start-page: 277
  volume-title: The Cambridge handbook of multimedia learning
  year: 2022
  ident: e_1_3_2_15_1
– ident: e_1_3_2_8_1
  doi: 10.1016/j.compedu.2017.07.003
– start-page: 450
  volume-title: The Cambridge handbook of multimedia learning
  year: 2022
  ident: e_1_3_2_55_1
– ident: e_1_3_2_37_1
  doi: 10.1037/0022-0663.91.4.638
– ident: e_1_3_2_53_1
  doi: 10.1080/15391523.2014.888265
– ident: e_1_3_2_27_1
  doi: 10.1145/258549.258797
– ident: e_1_3_2_51_1
  doi: 10.1037/0033-295X.110.1.145
– ident: e_1_3_2_29_1
  doi: 10.1016/j.learninstruc.2018.08.002
– ident: e_1_3_2_41_1
  doi: 10.1037/0022-0663.92.4.724
SSID ssj0002053
Score 2.4209347
Snippet The goal of this study was to determine whether learning processes and outcomes are affected by the emotional tone of the computer-generated voice in a...
SourceID crossref
informaworld
SourceType Enrichment Source
Index Database
Publisher
StartPage 5881
SubjectTerms computer-generated voice
emotion
multimedia learning
narrated slideshow
Text-to-speech
Title Limitations of Disembodied Computer-Generated Voice to Convey Emotion in Multimedia Lessons
URI https://www.tandfonline.com/doi/abs/10.1080/10447318.2024.2371681
Volume 41
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV29TsMwELZQWVj4R5Q_eWB1ldiOU48IWlUIOlEEElJkO7ZUCRLUBiTehmfhyTjHSdUOwMCSDPZZydk-f2fd3YfQOVh9l6dcEJcbQbgQEWwpqgl8NDeRyBmt6YBux2I04dcPSRtNOG_CKr0P7UKhiNpW-82t9LyNiIM35ymsRfDuKO9RBpDfJ1-vUwCKPqovehwtjDGNkhBjzznxMm0Sz0_DrBxPK8VLl46d4TbabPAivggTvIPWbLGLtlouBtxszT30VGcqhes3XDp8NZ3bF13mgDBx25uEGtOAMfF9CQYCVyW0Fe_2Aw8Cmw-eFl-fdU5unVGCb8AOwoD7aDIc3F2OSMOcQAzAtYrEqVUuzRUTwklBrZa2byLttE5yDk4e44aKVMexYkr2ldTg1gjmJLg3uWHSsgPUKcrCHiIsfVcDxxxLBI-16EOrTRXgTBBSynYRbxWWmeY_PbvFcxY31UdbPWdez1mj5y7qLcReQ12NvwTk8mxkVX2h4QL7SMZ-lT36h-wx2qCe8tcXbJUnqFPN3uwp4JBKn9UrDZ4sGn8DiUfSOg
linkProvider Taylor & Francis
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV29TsMwELZQGWDhH1F-PbC6SmzHiUcErQq0nVpUJKQoTmypEiQIAhJvw7PwZJzjpGoHYGDKYJ_lnO3zdyfffQidg9U3WcgFMVkqCBfCgyNFFYFJ89QTGaMVHdBwJPoTfjMNpgu5MPZZpfWhjSsUUdlqe7htMLp5EgdfzkPYjODeUd6hDDC_zb5eDSLwJmBPe_f9uTWmXuAe2XNOrEyTxfPTMEv301L10oV7p7eFNmrAiC_cCm-jFZ3voM2GjAHXZ3MXPVSpSi7-hguDr2av-kkVGUBM3PQmrsg0gEx8V4CFwGUBbfm7_sBdR-eDZ_nXZ5WUW6WU4AEYQhhwD0163fFln9TUCSQFvFYSP9SJCbOECWGkoFpJHaWeMkoFGQcvj_GUilD5fsISGSVSgV8jmJHg32Qpk5rto1Ze5PoAYWm7pnDPsUBwX4kIWnWYANAEoSTRbcQbhcVp_Z-W3uIx9uvyo42eY6vnuNZzG3XmYs-usMZfAnJxNeKyimgYRz8Ss19lD_8he4bW-uPhIB5cj26P0Dq1_L-2eqs8Rq3y5U2fACgp1Wm1674B_s_U0w
linkToPdf http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LSwMxEA5SQbxYn1ifOXjdsrvJZpuj2JaqtXiw4kFYNtkEirpb7FbQX-Nv8Zc52eyWVlAPPeWQTMhz8k2Y-QahM9D6Ogkpc3QimUMZc-FK-cKBQVPpsoT4RTqgmwHrDenVQ1B5E05Kt0pjQ2tLFFHoanO5x4muPOKgpDSEswjWnU-bPgHIb4KvVxnAE-PVR9zBTBn7bmB97Cl1jEwVxPNbNwvP0wJ56dyz060jUQ3Yeps8Nae5aMqPH1yOS81oE22UoBSf21O0hVZUuo3qVcIHXN7_HfRYhEPZPz6cadweTdSLyBKAsbhq7VgiawCy-D4DLYTzDOrSN_WOOzZlEB6lX59F4G8RtoL7oGyhw1007HbuLnpOmZ7BkYAJc8cLVazDJCaMac58JbhqSVdoIYKEgiVJqPRZKDwvJjFvxVyA7cSI5mBDJZJwRfZQLc1StY8wN00lvKUkYNQTrAW1KowBzIJQHKsGotWuRLKcp0mh8Rx5JcVptYKRWcGoXMEGas7Expa84z8BPr_lUV78mmib4iQif8oeLCF7itZu292ofzm4PkTrvkkxbAhi-RGq5a9TdQy4Jxcnxcn-BqRI83s
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Limitations+of+Disembodied+Computer-Generated+Voice+to+Convey+Emotion+in%C2%A0Multimedia+Lessons&rft.jtitle=International+journal+of+human-computer+interaction&rft.au=Zhao%2C+Fangzheng&rft.au=Mayer%2C+Richard+E.&rft.date=2025-05-19&rft.pub=Taylor+%26+Francis&rft.issn=1044-7318&rft.eissn=1532-7590&rft.volume=41&rft.issue=10&rft.spage=5881&rft.epage=5893&rft_id=info:doi/10.1080%2F10447318.2024.2371681&rft.externalDBID=0YH&rft.externalDocID=2371681
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1044-7318&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1044-7318&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1044-7318&client=summon