A benchmark study on sentiment analysis for software engineering research

A recent research trend has emerged to identify developers' emotions, by applying sentiment analysis to the content of communication traces left in collaborative development environments. Trying to overcome the limitations posed by using off-the-shelf sentiment analysis tools, researchers recen...

Full description

Saved in:
Bibliographic Details
Published in2018 IEEE/ACM 15th International Conference on Mining Software Repositories (MSR) pp. 364 - 375
Main Authors Novielli, Nicole, Girardi, Daniela, Lanubile, Filippo
Format Conference Proceeding
LanguageEnglish
Published New York, NY, USA ACM 28.05.2018
SeriesACM Conferences
Subjects
Benchmark testing
Collaboration
Communication Channels
Human-centered computing > Collaborative and social computing > Collaborative and social computing theory, concepts and paradigms
Human-centered computing > Collaborative and social computing > Collaborative and social computing theory, concepts and paradigms > Computer supported cooperative work
Information systems > Information retrieval > Retrieval tasks and goals
Information systems > Information retrieval > Retrieval tasks and goals > Sentiment analysis
Labeling
NLP
Sentiment analysis
Social Software Engineering
Software
Software and its engineering > Software creation and management > Collaboration in software development
Software and its engineering > Software creation and management > Collaboration in software development > Programming teams
Software engineering
social software engineering
NLP
sentiment analysis
communication channels
Online AccessGet full text
ISBN9781450357166
1450357164
ISSN2574-3864
DOI10.1145/3196398.3196403

Cover

Abstract A recent research trend has emerged to identify developers' emotions, by applying sentiment analysis to the content of communication traces left in collaborative development environments. Trying to overcome the limitations posed by using off-the-shelf sentiment analysis tools, researchers recently started to develop their own tools for the software engineering domain. In this paper, we report a benchmark study to assess the performance and reliability of three sentiment analysis tools specifically customized for software engineering. Furthermore, we offer a reflection on the open challenges, as they emerge from a qualitative analysis of misclassified texts.1
AbstractList A recent research trend has emerged to identify developers' emotions, by applying sentiment analysis to the content of communication traces left in collaborative development environments. Trying to overcome the limitations posed by using off-the-shelf sentiment analysis tools, researchers recently started to develop their own tools for the software engineering domain. In this paper, we report a benchmark study to assess the performance and reliability of three sentiment analysis tools specifically customized for software engineering. Furthermore, we offer a reflection on the open challenges, as they emerge from a qualitative analysis of misclassified texts.1
A recent research trend has emerged to identify developers' emotions, by applying sentiment analysis to the content of communication traces left in collaborative development environments. Trying to overcome the limitations posed by using off-the-shelf sentiment analysis tools, researchers recently started to develop their own tools for the software engineering domain. In this paper, we report a benchmark study to assess the performance and reliability of three sentiment analysis tools specifically customized for software engineering. Furthermore, we offer a reflection on the open challenges, as they emerge from a qualitative analysis of misclassified texts.
Author Lanubile, Filippo
Novielli, Nicole
Girardi, Daniela
Author_xml – sequence: 1
  givenname: Nicole
  surname: Novielli
  fullname: Novielli, Nicole
  email: nicole.novielli@uniba.it
  organization: University of Bari Aldo Moro, Italy
– sequence: 2
  givenname: Daniela
  surname: Girardi
  fullname: Girardi, Daniela
  email: daniela.girardi@uniba.it
  organization: University of Bari Aldo Moro, Italy
– sequence: 3
  givenname: Filippo
  surname: Lanubile
  fullname: Lanubile, Filippo
  email: filippo.lanubile@uniba.it
  organization: University of Bari Aldo Moro, Italy
BookMark eNqNkD1PwzAQhs2XRCmdGVg8sqT47OTsjFXFR6VKLDBbjnNpQ1sH2UGo_55U7cTEcu9Jz6vT6blhl6ELxNgdiClAXjwqKFGVZnrIXKgzNim1GYBQhQbEczaShc4zZTC_-MOu2SSlTyGERJMD6BFbzHhFwa93Lm546r_rPe8CTxT6djcM7oLb7lObeNNFnrqm_3GROIVVG4hiG1Y8UiIX_fqWXTVum2hyyjH7eH56n79my7eXxXy2zJxS2GdS1cY0SN75Emuq_LA45Z2RiIIAtSm99whOYF1IrXMUXgI4ELoGhFyN2f3xbktE9iu2w-t7a4qykFIM9OFInd_Zqus2yYKwB2_25M2evA3V6T-rtootNeoXZx9qug
CODEN IEEPAD
ContentType Conference Proceeding
Copyright 2018 ACM
Copyright_xml – notice: 2018 ACM
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1145/3196398.3196403
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
Text complet a IEEE Electronic Library
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList

Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 9781450357166
1450357164
EISSN 2574-3864
EndPage 375
ExternalDocumentID 8595220
Genre orig-research
GroupedDBID 6IE
6IF
6IL
6IN
AAJGR
ABLEC
ACM
ADPZR
ALMA_UNASSIGNED_HOLDINGS
APO
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
GUFHI
IEGSK
LHSKQ
OCL
RIB
RIC
RIE
RIL
AAWTH
ADZIZ
CHZPO
ID FETCH-LOGICAL-a336t-23d88f6ecac96debccaca3ca82660e16789ccc61a06d5277460c211a107d16143
IEDL.DBID RIE
ISBN 9781450357166
1450357164
IngestDate Wed Aug 27 02:59:15 EDT 2025
Fri Sep 13 11:04:49 EDT 2024
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed false
IsScholarly true
Keywords social software engineering
NLP
sentiment analysis
communication channels
Language English
License Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org
LinkModel DirectLink
MeetingName ICSE '18: 40th International Conference on Software Engineering
MergedId FETCHMERGED-LOGICAL-a336t-23d88f6ecac96debccaca3ca82660e16789ccc61a06d5277460c211a107d16143
OpenAccessLink http://hdl.handle.net/11586/218922
PageCount 12
ParticipantIDs ieee_primary_8595220
acm_books_10_1145_3196398_3196403
acm_books_10_1145_3196398_3196403_brief
PublicationCentury 2000
PublicationDate 20180528
2018-May
PublicationDateYYYYMMDD 2018-05-28
2018-05-01
PublicationDate_xml – month: 05
  year: 2018
  text: 20180528
  day: 28
PublicationDecade 2010
PublicationPlace New York, NY, USA
PublicationPlace_xml – name: New York, NY, USA
PublicationSeriesTitle ACM Conferences
PublicationTitle 2018 IEEE/ACM 15th International Conference on Mining Software Repositories (MSR)
PublicationTitleAbbrev MSR
PublicationYear 2018
Publisher ACM
Publisher_xml – name: ACM
SSID ssj0002684117
ssj0003211714
Score 2.407457
Snippet A recent research trend has emerged to identify developers' emotions, by applying sentiment analysis to the content of communication traces left in...
SourceID ieee
acm
SourceType Publisher
StartPage 364
SubjectTerms Benchmark testing
Collaboration
Communication Channels
Human-centered computing -- Collaborative and social computing -- Collaborative and social computing theory, concepts and paradigms
Human-centered computing -- Collaborative and social computing -- Collaborative and social computing theory, concepts and paradigms -- Computer supported cooperative work
Information systems -- Information retrieval -- Retrieval tasks and goals
Information systems -- Information retrieval -- Retrieval tasks and goals -- Sentiment analysis
Labeling
NLP
Sentiment analysis
Social Software Engineering
Software
Software and its engineering -- Software creation and management -- Collaboration in software development
Software and its engineering -- Software creation and management -- Collaboration in software development -- Programming teams
Software engineering
Title A benchmark study on sentiment analysis for software engineering research
URI https://ieeexplore.ieee.org/document/8595220
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3dS8MwED-2Pfk0dRPnFxEEX2zXzzR9VHGoMBHmYG8lSVOUsU5qh-hf7yVt5weCvqUllHB3zd0vufsdwEnsiThjfmYJmUoL_bGyYtPNnbGYOVQobq5ixnf0ehrczsJZC87WtTBKKZN8pmw9NHf56VKu9FHZUHNxeR4C9DaaWVWrtT5P0awlTc2kfvYR2URuULP5uEE4NMYWM9twUOkmWW0uF9-aqhifMurCuFlNlUoyt1elsOX7D6LG_y53E_qf1Xvkfu2XtqCl8m3oNu0bSP039-DmnFzg6HHBiznR-YRvZJmTiU4f0h8mDV8JwbiWTHC7fuWFIl8IDEmTtteH6ejq4fLaqjsrWNz3aWl5fspYRpXkMqapEqhGyX3JEWtQR7nowGIpJXW5Q9PQwwiROhLlyRErphgiBv4OdPJlrnaBOCECjsgLBWUqYAiuokhIVyrcN1LcPoIBHKOYEw0ZXpKqCjpMalUktSoGcPrnnEQUTyobQE_LOXmuqDiSWsR7v7_ehw2Mb1iVn3gAnbJYqUOMIUpxZIznA5wpvoo
linkProvider IEEE
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3dS8MwED_mfNCnqZs4PyMIvtjazyx9VHFsug1hG-ytJGmGMtZJ7RD96730Y34g6FtaQgl36d39krvfAZwFjgimzJ0aQkbSQH-sjCDr5s5YwCwqFM-uYvoD2hl7dxN_UoGLVS2MUipLPlOmHmZ3-dFCLvVR2aXm4nIcBOjr6Pc9P6_WWp2oaN6SsmpSP7uIbVq2V_D52J5_mW23gJkZC5Vuk7XG5fxbW5XMq7Rr0C_XkyeTzMxlKkz5_oOq8b8L3oLGZ_0eeVh5pm2oqHgHamUDB1L8z3XoXpFrHD3OeTIjOqPwjSxiMtQJRPrDpGQsIRjZkiEa7FeeKPKFwpCUiXsNGLdvRzcdo-itYHDXpanhuBFjU6oklwGNlEBFSu5KjmiDWspGFxZIKanNLRr5DsaI1JIoT45oMcIg0XN3oRovYrUHxPIRcrQcX1CmPIbwqtUS0pYKLUeEBsRrwimKOdSg4SXM66D9sFBFWKiiCed_zglF8qSmTahrOYfPORlHWIh4__fXJ7DRGfV7Ya87uD-ATYx2WJ6teAjVNFmqI4woUnGcbaQPyiHB1w
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2018+IEEE%2FACM+15th+International+Conference+on+Mining+Software+Repositories+%28MSR%29&rft.atitle=A+Benchmark+Study+on+Sentiment+Analysis+for+Software+Engineering+Research&rft.au=Novielli%2C+Nicole&rft.au=Girardi%2C+Daniela&rft.au=Lanubile%2C+Filippo&rft.date=2018-05-01&rft.pub=ACM&rft.eissn=2574-3864&rft.spage=364&rft.epage=375&rft_id=info:doi/10.1145%2F3196398.3196403&rft.externalDocID=8595220
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781450357166/lc.gif&client=summon&freeimage=true
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781450357166/mc.gif&client=summon&freeimage=true
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781450357166/sc.gif&client=summon&freeimage=true