Speech Synthesis for the Generation of Artificial Personality

A synthetic voice personifies the system using it. In this work we examine the impact text content, voice quality and synthesis system have on the perceived personality of two synthetic voices. Subjects rated synthetic utterances based on the Big-Five personality traits and naturalness. The naturaln...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on affective computing Vol. 11; no. 2; pp. 361 - 372
Main Authors Aylett, Matthew P., Vinciarelli, Alessandro, Wester, Mirjam
Format Journal Article
LanguageEnglish
Published Piscataway IEEE 01.04.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Online AccessGet full text
ISSN1949-3045
1949-3045
DOI10.1109/TAFFC.2017.2763134

Cover

Abstract A synthetic voice personifies the system using it. In this work we examine the impact text content, voice quality and synthesis system have on the perceived personality of two synthetic voices. Subjects rated synthetic utterances based on the Big-Five personality traits and naturalness. The naturalness rating of synthesis output did not correlate significantly with any Big-Five characteristic except for a marginal correlation with openness. Although text content is dominant in personality judgments, results showed that voice quality change implemented using a unit selection synthesis system significantly affected the perception of the Big-Five, for example tense voice being associated with being disagreeable and lax voice with lower conscientiousness. In addition a comparison between a parametric implementation and unit selection implementation of the same voices showed that parametric voices were rated as significantly less neurotic than both the text alone and the unit selection system, while the unit selection was rated as more open than both the text alone and the parametric system. The results have implications for synthesis voice and system type selection for applications such as personal assistants and embodied conversational agents where developing an emotional relationship with the user, or developing a branding experience is important.
AbstractList A synthetic voice personifies the system using it. In this work we examine the impact text content, voice quality and synthesis system have on the perceived personality of two synthetic voices. Subjects rated synthetic utterances based on the Big-Five personality traits and naturalness. The naturalness rating of synthesis output did not correlate significantly with any Big-Five characteristic except for a marginal correlation with openness. Although text content is dominant in personality judgments, results showed that voice quality change implemented using a unit selection synthesis system significantly affected the perception of the Big-Five, for example tense voice being associated with being disagreeable and lax voice with lower conscientiousness. In addition a comparison between a parametric implementation and unit selection implementation of the same voices showed that parametric voices were rated as significantly less neurotic than both the text alone and the unit selection system, while the unit selection was rated as more open than both the text alone and the parametric system. The results have implications for synthesis voice and system type selection for applications such as personal assistants and embodied conversational agents where developing an emotional relationship with the user, or developing a branding experience is important.
Author Vinciarelli, Alessandro
Aylett, Matthew P.
Wester, Mirjam
Author_xml – sequence: 1
  givenname: Matthew P.
  orcidid: 0000-0001-7057-0525
  surname: Aylett
  fullname: Aylett, Matthew P.
  email: matthewaylett@gmail.com
  organization: School of Informatics, University of Edinburgh, Edinburgh, United Kingdom
– sequence: 2
  givenname: Alessandro
  orcidid: 0000-0002-9048-0524
  surname: Vinciarelli
  fullname: Vinciarelli, Alessandro
  email: vincia@dcs.gla.ac.uk
  organization: Computing Sciences, University of Glasgow, Glasgow, United Kingdom
– sequence: 3
  givenname: Mirjam
  orcidid: 0000-0002-3199-0081
  surname: Wester
  fullname: Wester, Mirjam
  email: mwester@inf.ed.ac.uk
  organization: School of Informatics, University of Edinburgh, Edinburgh, United Kingdom
BookMark eNp9kEtPAjEUhRuDiYj8Ad00cT3Yx8y0XbggRMCERBNw3ZTOnVAyTrEtC_69wyPGuPBu7lmc7z7OLeq1vgWE7ikZUUrU02o8nU5GjFAxYqLklOdXqE9VrjJO8qL3S9-gYYxb0hXnvGSij56XOwC7wctDmzYQXcS1D7iTeAYtBJOcb7Gv8TgkVzvrTIPfIUTfmsalwx26rk0TYXjpA_QxfVlN5tnibfY6GS8yy7lK2brbbyvgVkojDBV1vaZKWqOKqrAmJ2DXhgvLqKpAkpyyylBLCQdblMqKig_Q43nuLvivPcSkt34fuhuiZjmRZSFFSTuXPLts8DEGqLV16fRBCsY1mhJ9zEuf8tLHvPQlrw5lf9BdcJ8mHP6HHs6QA4AfQJJSMpHzb7o6eBk
CODEN ITACBQ
CitedBy_id crossref_primary_10_1016_j_tics_2025_01_010
crossref_primary_10_1109_MCE_2022_3180183
crossref_primary_10_1080_17517575_2023_2246188
crossref_primary_10_1007_s12369_021_00801_w
crossref_primary_10_1109_JPROC_2023_3261137
crossref_primary_10_1109_TAFFC_2019_2930695
crossref_primary_10_3389_fnbot_2020_593732
crossref_primary_10_1007_s12193_018_0270_6
crossref_primary_10_1007_s11365_022_00823_4
crossref_primary_10_3390_s24227151
crossref_primary_10_1080_07434618_2023_2262032
crossref_primary_10_1016_j_chb_2023_107788
Cites_doi 10.2307/2087389
10.1002/ejsp.2420080405
10.1086/431246
10.1109/TSA.2005.855840
10.1146/annurev.psych.57.102904.190127
10.1207/s15327957pspr0803_3
10.2307/2786183
10.21437/Interspeech.2016-1188
10.1037/0022-3514.38.2.270
10.1037/0033-2909.116.2.245
10.1007/978-3-540-74997-4_65
10.1093/oso/9780199211425.001.0001
10.1109/TAFFC.2014.2330816
10.1146/annurev.psych.59.103006.093707
10.1016/j.specom.2009.04.004
10.1006/jpho.2001.0147
10.1017/CBO9780511812743
10.1109/TASL.2013.2269291
10.1007/978-3-642-15892-6_20
10.1037/0033-2909.115.1.153
10.1109/ICHR.2005.1573596
10.1016/j.csl.2004.03.003
10.1146/annurev.psych.52.1.197
10.1109/TASL.2010.2045239
10.1109/ICASSP.1983.1172250
10.1177/0261927X09351676
10.1121/1.398894
10.21437/SSW.2016-33
10.1016/j.imavis.2008.11.007
10.21437/Interspeech.2016-290
10.1109/T-AFFC.2011.38
10.3989/loquens.2014.006
10.1080/01621459.1986.10478364
10.1016/S0167-6393(98)00085-5
10.1037/1076-898X.7.3.171
10.1109/TASLP.2014.2385478
10.1007/s11370-008-0017-4
10.1016/S0167-6393(02)00082-1
10.1126/science.283.5406.1272
10.1017/CBO9780511596544.009
10.1017/CBO9780511596544
10.1017/CBO9780511596544.012
10.1145/257089.257305
10.1109/JSTSP.2014.2307274
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
8FD
JQ2
L7M
L~C
L~D
DOI 10.1109/TAFFC.2017.2763134
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Xplore
CrossRef
Computer and Information Systems Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Computer and Information Systems Abstracts
Technology Research Database
Computer and Information Systems Abstracts – Academic
Advanced Technologies Database with Aerospace
ProQuest Computer Science Collection
Computer and Information Systems Abstracts Professional
DatabaseTitleList Computer and Information Systems Abstracts

Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Xplore
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Psychology
Computer Science
EISSN 1949-3045
EndPage 372
ExternalDocumentID 10_1109_TAFFC_2017_2763134
8068274
Genre orig-research
GrantInformation_xml – fundername: EPSRC
  grantid: EP/N035305/1
– fundername: Royal Society through a Royal Society Industrial Fellowship
– fundername: European Union's Horizon 2020 research and innovation programme
  grantid: 645378
GroupedDBID 0R~
4.4
5VS
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABJNI
ABQJQ
ABVLG
AENEX
AGQYO
AGSQL
AHBIQ
AKJIK
AKQYR
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
EBS
EJD
HZ~
IEDLZ
IFIPE
IPLJI
JAVBF
M43
O9-
OCL
PQQKQ
RIA
RIE
RNI
RZB
AAYXX
CITATION
7SC
8FD
JQ2
L7M
L~C
L~D
ID FETCH-LOGICAL-c339t-b949cde3c88a7a17ffb198ca95d5ca40ecba37c219de80412da1c103ec569c7d3
IEDL.DBID RIE
ISSN 1949-3045
IngestDate Mon Jun 30 06:21:28 EDT 2025
Tue Jul 01 02:57:51 EDT 2025
Thu Apr 24 23:01:10 EDT 2025
Wed Aug 27 02:38:30 EDT 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 2
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c339t-b949cde3c88a7a17ffb198ca95d5ca40ecba37c219de80412da1c103ec569c7d3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0002-3199-0081
0000-0001-7057-0525
0000-0002-9048-0524
OpenAccessLink https://www.research.ed.ac.uk/en/publications/c0583a49-f6f8-431f-895c-93497cd207dc
PQID 2408658761
PQPubID 2040414
PageCount 12
ParticipantIDs crossref_primary_10_1109_TAFFC_2017_2763134
proquest_journals_2408658761
ieee_primary_8068274
crossref_citationtrail_10_1109_TAFFC_2017_2763134
ProviderPackageCode CITATION
AAYXX
PublicationCentury 2000
PublicationDate 2020-04-01
PublicationDateYYYYMMDD 2020-04-01
PublicationDate_xml – month: 04
  year: 2020
  text: 2020-04-01
  day: 01
PublicationDecade 2020
PublicationPlace Piscataway
PublicationPlace_xml – name: Piscataway
PublicationTitle IEEE transactions on affective computing
PublicationTitleAbbrev T-AFFC
PublicationYear 2020
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref57
clark (ref20) 2004
ref12
ref59
zen (ref23) 2013
ref58
ref14
krahmer (ref13) 2003
hu (ref31) 2013
kominek (ref21) 2005
ref52
hunt (ref19) 1996
ref55
ref11
ref54
scherer (ref1) 1979
bennett (ref56) 2006
ref17
ref16
reeves (ref4) 1996
ref50
ref45
ref48
ref47
laver (ref24) 1980; 31
ref41
ref44
ref49
saucier (ref42) 1996
ref8
ref7
wester (ref18) 2015
ref3
ref5
zen (ref22) 2007
ref40
howell (ref46) 2012
ref35
ref34
ref37
ref36
ref30
ref33
trouvain (ref10) 2006
ref32
ref2
ref39
ref38
oord (ref68) 2016
scrhöder (ref27) 2003
tapus (ref15) 2008
schmitz (ref9) 2007
aylett (ref28) 2008
watts (ref51) 2015
ref67
ref26
ref25
ref64
ref63
hofer (ref29) 2005
ref66
aylett (ref53) 2013
ref65
nass (ref6) 2005
ref60
ref62
ref61
nettle (ref43) 2007
References_xml – start-page: 2217
  year: 2015
  ident: ref51
  article-title: Sentence-level control vectors for deep neural network speech synthesis
  publication-title: Proc Annu Conf Int Speech Commun Assoc
– ident: ref62
  doi: 10.2307/2087389
– ident: ref61
  doi: 10.1002/ejsp.2420080405
– ident: ref65
  doi: 10.1086/431246
– year: 2006
  ident: ref10
  article-title: Modeling personality features by changing prosody in synthetic speech
  publication-title: presented at the Proc Speech Prosody
– start-page: 3365
  year: 2015
  ident: ref18
  article-title: Artificial personality and disfluency
  publication-title: Proc Annu Conf Int Speech Commun Assoc
– year: 2012
  ident: ref46
  publication-title: Statistical Methods for Psychology
– ident: ref55
  doi: 10.1109/TSA.2005.855840
– ident: ref40
  doi: 10.1146/annurev.psych.57.102904.190127
– start-page: 147
  year: 2004
  ident: ref20
  article-title: Festival 2 - build your own general purpose unit selection speech synthesiser
  publication-title: Proc 5th ISCA Workshop Speech Synthesis
– year: 2016
  ident: ref68
  article-title: WaveNet: A generative model for raw audio
– ident: ref45
  doi: 10.1207/s15327957pspr0803_3
– ident: ref63
  doi: 10.2307/2786183
– ident: ref67
  doi: 10.21437/Interspeech.2016-1188
– ident: ref2
  doi: 10.1037/0022-3514.38.2.270
– ident: ref44
  doi: 10.1037/0033-2909.116.2.245
– year: 2005
  ident: ref6
  publication-title: Wired for Speech How Voice Activates and Advances the Human-Computer Relationship
– ident: ref30
  doi: 10.1007/978-3-540-74997-4_65
– year: 2007
  ident: ref43
  publication-title: Personality What Makes You the Way You Are
  doi: 10.1093/oso/9780199211425.001.0001
– year: 2006
  ident: ref56
  article-title: The Blizzard challenge 2006
  publication-title: Proc Blizzard Challenge
– ident: ref5
  doi: 10.1109/TAFFC.2014.2330816
– start-page: 133
  year: 2013
  ident: ref53
  article-title: Expressive speech synthesis: Synthesising ambiguity
  publication-title: Proc ISCA Speech Synthesis Workshop
– ident: ref3
  doi: 10.1146/annurev.psych.59.103006.093707
– ident: ref32
  doi: 10.1016/j.specom.2009.04.004
– start-page: 155
  year: 2013
  ident: ref31
  article-title: An experimental comparison of multiple vocoder types
  publication-title: Proc ISCA Speech Synthesis Workshop
– ident: ref26
  doi: 10.1006/jpho.2001.0147
– ident: ref37
  doi: 10.1017/CBO9780511812743
– ident: ref33
  doi: 10.1109/TASL.2013.2269291
– start-page: 501
  year: 2005
  ident: ref29
  article-title: Informed blending of databases for emotional speech synthesis
  publication-title: Proc Annu Conf Int Speech Commun Assoc
– ident: ref11
  doi: 10.1007/978-3-642-15892-6_20
– start-page: 2589
  year: 2003
  ident: ref27
  article-title: Expressing vocal effort in concatenative synthesis
  publication-title: Proc Int Congr Phonetic Sci
– ident: ref58
  doi: 10.1037/0033-2909.115.1.153
– start-page: 7
  year: 2003
  ident: ref13
  article-title: Audio-visual cues to personality: An experimental approach
  publication-title: Proc AAMAS Workshop Embodied Agents Individuals
– ident: ref16
  doi: 10.1109/ICHR.2005.1573596
– ident: ref54
  doi: 10.1016/j.csl.2004.03.003
– ident: ref38
  doi: 10.1146/annurev.psych.52.1.197
– ident: ref48
  doi: 10.1109/TASL.2010.2045239
– ident: ref36
  doi: 10.1109/ICASSP.1983.1172250
– ident: ref47
  doi: 10.1177/0261927X09351676
– ident: ref60
  doi: 10.1121/1.398894
– start-page: 7962
  year: 2013
  ident: ref23
  article-title: Statistical parametric speech synthesis using deep neural networks
  publication-title: Proc IEEE Int Conf Acoust Speech Signal Process
– volume: 31
  start-page: 1
  year: 1980
  ident: ref24
  article-title: The phonetic description of voice quality
  publication-title: Cambridge Studies in Linguistics London
– ident: ref66
  doi: 10.21437/SSW.2016-33
– ident: ref59
  doi: 10.1016/j.imavis.2008.11.007
– ident: ref52
  doi: 10.21437/Interspeech.2016-290
– ident: ref12
  doi: 10.1109/T-AFFC.2011.38
– start-page: 192
  year: 1996
  ident: ref19
  article-title: Unit selection in concatanative speech synthesis using a large speech database
  publication-title: Proc Acoust Speech Signal Process
– start-page: 313
  year: 2007
  ident: ref9
  article-title: Modeling personality in voice of talking products through prosodic parameters
  publication-title: Proc Int Conf Internet of Things
– start-page: 294
  year: 2007
  ident: ref22
  article-title: The HMM-based speech synthesis system (HTS) version 2.0
  publication-title: Proc 6th ISCA Speech Synth Workshop
– ident: ref34
  doi: 10.3989/loquens.2014.006
– ident: ref57
  doi: 10.1080/01621459.1986.10478364
– start-page: 21
  year: 1996
  ident: ref42
  article-title: The language of personality: Lexical perspectives on the five-factor model
  publication-title: The Five-Factor Model of Personality
– ident: ref35
  doi: 10.1016/S0167-6393(98)00085-5
– ident: ref8
  doi: 10.1037/1076-898X.7.3.171
– ident: ref50
  doi: 10.1109/TASLP.2014.2385478
– year: 1996
  ident: ref4
  publication-title: The Media Equation How People Treat Computers Television and New Media Like Real People and Places
– ident: ref14
  doi: 10.1007/s11370-008-0017-4
– start-page: 133
  year: 2008
  ident: ref15
  article-title: Socially assistive robots: The link between personality, empathy, physiological signals, and task performance
  publication-title: Proc AAAI Spring Symp
– ident: ref25
  doi: 10.1016/S0167-6393(02)00082-1
– ident: ref64
  doi: 10.1126/science.283.5406.1272
– ident: ref41
  doi: 10.1017/CBO9780511596544.009
– ident: ref39
  doi: 10.1017/CBO9780511596544
– year: 2008
  ident: ref28
  article-title: Adding and controlling emotion in synthesised speech
– start-page: 85
  year: 2005
  ident: ref21
  article-title: The Blizzard challenge 2005 CMU entry - a method for improving speech synthesis systems
  publication-title: Proc Annu Conf Int Speech Commun Assoc
– ident: ref17
  doi: 10.1017/CBO9780511596544.012
– start-page: 147
  year: 1979
  ident: ref1
  article-title: Personality markers in speech
  publication-title: Social markers in speech
– ident: ref7
  doi: 10.1145/257089.257305
– ident: ref49
  doi: 10.1109/JSTSP.2014.2307274
SSID ssj0000333627
Score 2.3196967
Snippet A synthetic voice personifies the system using it. In this work we examine the impact text content, voice quality and synthesis system have on the perceived...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 361
SubjectTerms automatic personality perception
automatic personality recognition
automatic personality synthesis
Computational modeling
Digital signal processing
Hidden Markov models
Personality
Psychology
Robots
Speech
Speech recognition
Speech synthesis
Title Speech Synthesis for the Generation of Artificial Personality
URI https://ieeexplore.ieee.org/document/8068274
https://www.proquest.com/docview/2408658761
Volume 11
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LSwMxEB5qT734qmJ9kYM33brZpJvNwUMRiwiKUAVvS3Z2FkVpi20P-utNsg_xgXjLIVnCvCc78w3AkVYkSEcUiDjMA2kKHmQyxEAURhIn77JdtcVNfHkvrx4GDy04aXphiMgXn1HfLf2__HyKS_dUdpqEcWKzqBVYsWJW9mo17ymhENYWq7ovJtSnd8PR6NwVb6l-ZLWIC_nF9_hhKj8ssHcrozW4ri9UVpM895eLrI_v37Aa_3vjdVit4ks2LAViA1o02YS1enYDq1R5EzqN5Xvrwtl4RoSPbPw2sfHg_GnObCjL7JKVqNSOeWxa-K-WiBPs9jOI34L70cXd-WVQzVUIUAi9CDItNeYkMEmMMlwVRcZ1gkYP8gEaGRJmRii0tiwnB08U5YYjDwXhINaocrEN7cl0QjvAZGJsfqh4hMZqPxZaZpxnhT3hgNwIe8BriqdYgY672RcvqU8-Qp16LqWOS2nFpR4cN2dmJeTGn7u7juzNzoriPdivGZtWWjlPHZybjbhUzHd_P7UHncjl074yZx_ai9clHdigY5Edemn7AFc21AA
linkProvider IEEE
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LTxsxEB7xOMAFyktNS6kP3Ogm67V3vT70gCKitDyERCLltvLOzgpElUQkOdBfX9v7qAoV4uaDZ2XNeF7emW8ATrUiQTqiQCRhEUhT8iCXIQaiNJI4eZftqi1ukuFY_pzEkzX41vbCEJEvPqOuW_p_-cUMV-6prJeGSWqzqHXYtH5fxlW3VvuiEgphrbFqOmNC3RudDwZ9V76lupHVIy7kP97Hj1N5ZYO9YxnswnVzpKqe5LG7WuZd_P0CrfG9Z_4AO3WEyc6rK7EHazTdh91megOrlXkftlvb93wA3-_mRHjP7p6nNiJcPCyYDWaZXbIKl9qJj81K_9UKc4Ld_g3jD2E8uBj1h0E9WSFAIfQyyLXUWJDANDXKcFWWOdcpGh0XMRoZEuZGKLTWrCAHUBQVhiMPBWGcaFSFOIKN6WxKH4HJ1NgMUfEIjdV_LLXMOc9LS-Gg3Ag7wBuOZ1jDjrvpF78yn36EOvNSypyUslpKHThraeYV6Mabuw8c29udNcc7cNwINqv1cpE5QDcbc6mEf_o_1VfYGo6ur7KrHzeXn2E7ctm1r9M5ho3l04q-2BBkmZ_4m_cHmjrXTQ
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Speech+Synthesis+for+the+Generation+of+Artificial+Personality&rft.jtitle=IEEE+transactions+on+affective+computing&rft.au=Aylett%2C+Matthew+P.&rft.au=Vinciarelli%2C+Alessandro&rft.au=Wester%2C+Mirjam&rft.date=2020-04-01&rft.issn=1949-3045&rft.eissn=1949-3045&rft.volume=11&rft.issue=2&rft.spage=361&rft.epage=372&rft_id=info:doi/10.1109%2FTAFFC.2017.2763134&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TAFFC_2017_2763134
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1949-3045&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1949-3045&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1949-3045&client=summon