A Longitudinal Study of Static Analysis Warning Evolution and the Effects of PMD on Software Quality in Apache Open Source Projects

Automated static analysis tools (ASATs) have become a major part of the software development workflow. Acting on the generated warnings, i.e., changing the code indicated in the warning, should be part of, at latest, the code review phase. Despite this being a best practice in software development,...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Trautsch, Alexander, Herbold, Steffen, Grabowski, Jens
Format Paper Journal Article
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 27.08.2020
Subjects
Best practice
Building codes
Computer Science - Software Engineering
Open source software
Software development
Source code
Trends
Warning
Workflow
Online AccessGet full text

Cover

Abstract Automated static analysis tools (ASATs) have become a major part of the software development workflow. Acting on the generated warnings, i.e., changing the code indicated in the warning, should be part of, at latest, the code review phase. Despite this being a best practice in software development, there is still a lack of empirical research regarding the usage of ASATs in the wild. In this work, we want to study ASAT warning trends in software via the example of PMD as an ASAT and its usage in open source projects. We analyzed the commit history of 54 projects (with 112,266 commits in total), taking into account 193 PMD rules and 61 PMD releases. We investigate trends of ASAT warnings over up to 17 years for the selected study subjects regarding changes of warning types, short and long term impact of ASAT use, and changes in warning severities. We found that large global changes in ASAT warnings are mostly due to coding style changes regarding braces and naming conventions. We also found that, surprisingly, the influence of the presence of PMD in the build process of the project on warning removal trends for the number of warnings per lines of code is small and not statistically significant. Regardless, if we consider defect density as a proxy for external quality, we see a positive effect if PMD is present in the build configuration of our study subjects.
AbstractList Empirical Software Engineering 25 (2020) 5137-5192 Automated static analysis tools (ASATs) have become a major part of the software development workflow. Acting on the generated warnings, i.e., changing the code indicated in the warning, should be part of, at latest, the code review phase. Despite this being a best practice in software development, there is still a lack of empirical research regarding the usage of ASATs in the wild. In this work, we want to study ASAT warning trends in software via the example of PMD as an ASAT and its usage in open source projects. We analyzed the commit history of 54 projects (with 112,266 commits in total), taking into account 193 PMD rules and 61 PMD releases. We investigate trends of ASAT warnings over up to 17 years for the selected study subjects regarding changes of warning types, short and long term impact of ASAT use, and changes in warning severities. We found that large global changes in ASAT warnings are mostly due to coding style changes regarding braces and naming conventions. We also found that, surprisingly, the influence of the presence of PMD in the build process of the project on warning removal trends for the number of warnings per lines of code is small and not statistically significant. Regardless, if we consider defect density as a proxy for external quality, we see a positive effect if PMD is present in the build configuration of our study subjects.
Automated static analysis tools (ASATs) have become a major part of the software development workflow. Acting on the generated warnings, i.e., changing the code indicated in the warning, should be part of, at latest, the code review phase. Despite this being a best practice in software development, there is still a lack of empirical research regarding the usage of ASATs in the wild. In this work, we want to study ASAT warning trends in software via the example of PMD as an ASAT and its usage in open source projects. We analyzed the commit history of 54 projects (with 112,266 commits in total), taking into account 193 PMD rules and 61 PMD releases. We investigate trends of ASAT warnings over up to 17 years for the selected study subjects regarding changes of warning types, short and long term impact of ASAT use, and changes in warning severities. We found that large global changes in ASAT warnings are mostly due to coding style changes regarding braces and naming conventions. We also found that, surprisingly, the influence of the presence of PMD in the build process of the project on warning removal trends for the number of warnings per lines of code is small and not statistically significant. Regardless, if we consider defect density as a proxy for external quality, we see a positive effect if PMD is present in the build configuration of our study subjects.
Author Trautsch, Alexander
Grabowski, Jens
Herbold, Steffen
Author_xml – sequence: 1
  givenname: Alexander
  surname: Trautsch
  fullname: Trautsch, Alexander
– sequence: 2
  givenname: Steffen
  surname: Herbold
  fullname: Herbold, Steffen
– sequence: 3
  givenname: Jens
  surname: Grabowski
  fullname: Grabowski, Jens
BackLink https://doi.org/10.1007/s10664-020-09880-1$$DView published paper (Access to full text may be restricted)
https://doi.org/10.48550/arXiv.1912.02179$$DView paper in arXiv
BookMark eNotUMlOwzAQtRBIlNIP4IQlzile4iY5RqUsUlGLWolj5Dh2cRXsYCeFnPlxnJbDaEZvGc28K3BurJEA3GA0jVPG0D13P_owxRkmU0Rwkp2BEaEUR2lMyCWYeL9HCJFZQhijI_Cbw6U1O912lTa8hpsw9NCqMPBWC5gHsPfaw3fujDY7uDjYumu1NZCbCrYfEi6UkqL1g2n9-gADs7Gq_eZOwreO17rtoTYwb7gI4lUjB75zQsK1s_vBeQ0uFK-9nPz3Mdg-Lrbz52i5enqZ58uIM8KiOMYlU4hKGT7hmUywIJIFpMRSKMRkhSpSijSmqUCMZiWZobgqE8ZpWikW0zG4Pa09JlQ0Tn9y1xdDUsUxqaC4OykaZ7866dtiHy4NCfiCUEIIS0PRP-hEbgQ
ContentType Paper
Journal Article
Copyright 2020. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
http://arxiv.org/licenses/nonexclusive-distrib/1.0
Copyright_xml – notice: 2020. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
– notice: http://arxiv.org/licenses/nonexclusive-distrib/1.0
DBID 8FE
8FG
ABJCF
ABUWG
AFKRA
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
HCIFZ
L6V
M7S
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
AKY
GOX
DOI 10.48550/arxiv.1912.02179
DatabaseName ProQuest SciTech Collection
ProQuest Technology Collection
Materials Science & Engineering Collection
ProQuest Central (Alumni)
ProQuest Central UK/Ireland
ProQuest Central Essentials
ProQuest Central
Technology Collection
ProQuest One
ProQuest Central Korea
SciTech Premium Collection
ProQuest Engineering Collection
Engineering Database
ProQuest Central Premium
ProQuest One Academic (New)
Publicly Available Content Database
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering Collection
arXiv Computer Science
arXiv.org
DatabaseTitle Publicly Available Content Database
Engineering Database
Technology Collection
ProQuest One Academic Middle East (New)
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Engineering Collection
ProQuest One Academic UKI Edition
ProQuest Central Korea
Materials Science & Engineering Collection
ProQuest Central (New)
ProQuest One Academic
ProQuest One Academic (New)
Engineering Collection
DatabaseTitleList
Publicly Available Content Database
Database_xml – sequence: 1
  dbid: GOX
  name: arXiv.org
  url: http://arxiv.org/find
  sourceTypes: Open Access Repository
– sequence: 2
  dbid: 8FG
  name: ProQuest Technology Collection
  url: https://search.proquest.com/technologycollection1
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Physics
EISSN 2331-8422
ExternalDocumentID 1912_02179
Genre Working Paper/Pre-Print
GroupedDBID 8FE
8FG
ABJCF
ABUWG
AFKRA
ALMA_UNASSIGNED_HOLDINGS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
FRJ
HCIFZ
L6V
M7S
M~E
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
AKY
GOX
ID FETCH-LOGICAL-a525-441b5f03ee842a9e71c2e55f0b1ecf05ed0d2bc8438c0539b2604db75a38df543
IEDL.DBID BENPR
IngestDate Tue Jul 22 21:59:51 EDT 2025
Mon Jun 30 09:33:42 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a525-441b5f03ee842a9e71c2e55f0b1ecf05ed0d2bc8438c0539b2604db75a38df543
Notes SourceType-Working Papers-1
ObjectType-Working Paper/Pre-Print-1
content type line 50
OpenAccessLink https://www.proquest.com/docview/2322258225?pq-origsite=%requestingapplication%
PQID 2322258225
PQPubID 2050157
ParticipantIDs arxiv_primary_1912_02179
proquest_journals_2322258225
PublicationCentury 2000
PublicationDate 20200827
PublicationDateYYYYMMDD 2020-08-27
PublicationDate_xml – month: 08
  year: 2020
  text: 20200827
  day: 27
PublicationDecade 2020
PublicationPlace Ithaca
PublicationPlace_xml – name: Ithaca
PublicationTitle arXiv.org
PublicationYear 2020
Publisher Cornell University Library, arXiv.org
Publisher_xml – name: Cornell University Library, arXiv.org
SSID ssj0002672553
Score 1.734805
SecondaryResourceType preprint
Snippet Automated static analysis tools (ASATs) have become a major part of the software development workflow. Acting on the generated warnings, i.e., changing the...
Empirical Software Engineering 25 (2020) 5137-5192 Automated static analysis tools (ASATs) have become a major part of the software development workflow....
SourceID arxiv
proquest
SourceType Open Access Repository
Aggregation Database
SubjectTerms Best practice
Building codes
Computer Science - Software Engineering
Open source software
Software development
Source code
Trends
Warning
Workflow
SummonAdditionalLinks – databaseName: arXiv.org
  dbid: GOX
  link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwdV1LTwIxEG6AkxejUQOKZg5eV3f72MeRKGiMKIkYuW3abZtwWQgLKGf_uNPtEg_Ga9u5TKedb6bTbwi5ZojalElEICPDA67RjDGyjQMTSUSnMkaP4hL645f48Z0_zcSsRWD_F0auvuZbzw-sqlsMJuiNQ81Zm7QpdSVbD68z_zhZU3E163_XIcash_5crbW_GB2RwwbowcDvzDFpmfKEfA_geeE6BG2060YFropvBwsLDvPNC9hThMCHT1jAcNuYBmDEDwjWwNMNV05oMr4HnHnDm_RTrgx4PowdzEsYLB1RM7hyEZx3-XmY-JxLdUqmo-H07jFo2iAEUlARIF5RwobMmJRTmZkkKqgROKIiU9hQGB1qqoqUs7TAE5UpjFC4VomQLNVWcHZGOuWiNF0CVoZMSk55nDJuY5npwhrH8G55qDRNe6RbKy9feqaL3Ok1r_XaI_29PvPGyqucumcagRBDnP8veUEOqItRQzyRSZ901quNuURHvlZX9W7-AA3tngI
  priority: 102
  providerName: Cornell University
Title A Longitudinal Study of Static Analysis Warning Evolution and the Effects of PMD on Software Quality in Apache Open Source Projects
URI https://www.proquest.com/docview/2322258225
https://arxiv.org/abs/1912.02179
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV09T8MwELUoFRIbn2qhoBtYQ1PHTtIJFeiHEIUKiugWObEjdUlD0xa6sPDHuUtSGJBYIsWWl7N99-759I6xCwdRW2g8aamWEZbQeIwxs3Ut01KITpWLEYUI_eGDO3gRdxM5KQm3rCyr3PjE3FHrWUQceZPTk4DEcCav0jeLukbR62rZQqPCquiCfUy-qtfdh9HTD8vCXQ8xs1M8Z-biXU01_5iuLjFN4ZeEx6kQMR_644zzCNPbY9WRSs18n22Z5IDt5IWZUXbIvjpwP6OeQktN_auA6v7WMIuBUOI0go2oCLwWFAd0V-VhApVoQHgHhUBxRotGw1vAmWf0ve9qbqBQ0FjDNIFOStLOQAUmOE-MPowKliY7YuNed3wzsMrGCZaSXFqIcEIZ244xvuCqbbxWxI3EkbBlotiWRtuah5EvHD_CO9gOMacROvSkcnwdS-Ecs-1klpgag1jZjlKCC9d3ROyqto5iQ5rwsbBDzf06q-XGC9JCGyMguwa5XeussbFnUN6LLPjdxZP_p0_ZLqfM1sZ77DXY9mK-NGcY_hfhOav4vf55udP413-c4Hf42f0GfYazGw
linkProvider ProQuest
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV05T-tAEB7xQOi9jlPA45gCSoOzh-0UCCEgBEhQJIKgs9betZTGCTFXan4P_5EZO4YCiY52V3YxOzvzzbHfAOxKQm2JC7VnGk55ypIaU2QbeK5hCJ2agDwKJ_S710H7Vl3e6_sZeK_fwnBbZW0TS0NthynnyA8ElwQ0uTN9NHrweGoUV1frERqVWly5yQuFbMXhxSmd754QrbP-SdubThXwjBbaI_ef6MyXzkVKmKYLG6lwmlaShkszXzvrW5GkkZJRSgraTAjwK5uE2sjIZlpJ-u0fmFNSNvlCRa3zz5SOCEIC6LKqnZZMYQdm_Dp43qeYSOwz-Oeux3Lpm-Uv3VlrAeZ6ZuTGizDj8iWYL7tA02IZ3o6xM-QBRk-Wh2UhNxlOcJghQ9JBijWDCd5V-RQ8e55qLprcImFJrNiQC_6o1z1F2rkhQ_9ixg4ruo4JDnI8HjGPNHI3C-1z-QB7VUqoWIH-b8hzFWbzYe7WADPjS2OUUEEkVRaYpk0zxwT0mfITK6J1WCuFF48qIo6Y5RqXcl2HzVqe8fQSFvGXymz8vL0Df9v9bifuXFxf_Yd_gkNqnwxIuAmzj-Mnt0W44zHZLk8bIf5l7foAR-3qjw
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Longitudinal+Study+of+Static+Analysis+Warning+Evolution+and+the+Effects+of+PMD+on+Software+Quality+in+Apache+Open+Source+Projects&rft.jtitle=arXiv.org&rft.au=Trautsch%2C+Alexander&rft.au=Herbold%2C+Steffen&rft.au=Grabowski%2C+Jens&rft.date=2020-08-27&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.1912.02179