Software Numerical Instability Detection and Diagnosis by Combining Stochastic and Infinite-Precision Testing

Numerical instability is a well-known problem that may cause serious runtime failures. This paper discusses the reason of instability in software development process, and presents a toolchain that not only detects the potential instability in software, but also diagnoses the reason for such instabil...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on software engineering Vol. 43; no. 10; pp. 975 - 994
Main Authors Enyi Tang, Xiangyu Zhang, Muller, Norbert T., Zhenyu Chen, Xuandong Li
Format Journal Article
LanguageEnglish
Published New York IEEE 01.10.2017
IEEE Computer Society
Subjects
Algorithm design and analysis
Computer bugs
infinite-precision arithmetic
Libraries
Numerical analysis
Software
Software algorithms
Software development
Software testing
Stability
stochastic arithmetic
Online AccessGet full text
ISSN0098-5589
1939-3520
DOI10.1109/TSE.2016.2642956

Cover

Abstract Numerical instability is a well-known problem that may cause serious runtime failures. This paper discusses the reason of instability in software development process, and presents a toolchain that not only detects the potential instability in software, but also diagnoses the reason for such instability. We classify the reason of instability into two categories. When it is introduced by software requirements, we call the instability caused by problem . In this case, it cannot be avoided by improving software development, but requires inspecting the requirements, especially the underlying mathematical properties. Otherwise, we call the instability caused by practice. We design our toolchain as four loosely-coupled tools, which combine stochastic arithmetic with infinite-precision testing. Each tool in our toolchain can be configured with different strategies according to the properties of the analyzed software. We evaluate our toolchain on subjects from literature. The results show that it effectively detects and separates the instabilities caused by problems from others. We also conduct an evaluation on the latest version of GNU Scientific Library, and the toolchain finds a few real bugs in the well-maintained and widely deployed numerical library. With the help of our toolchain, we report the details and fixing advices to the GSL buglist.
AbstractList Numerical instability is a well-known problem that may cause serious runtime failures. This paper discusses the reason of instability in software development process, and presents a toolchain that not only detects the potential instability in software, but also diagnoses the reason for such instability. We classify the reason of instability into two categories. When it is introduced by software requirements, we call the instability caused by problem . In this case, it cannot be avoided by improving software development, but requires inspecting the requirements, especially the underlying mathematical properties. Otherwise, we call the instability caused by practice. We design our toolchain as four loosely-coupled tools, which combine stochastic arithmetic with infinite-precision testing. Each tool in our toolchain can be configured with different strategies according to the properties of the analyzed software. We evaluate our toolchain on subjects from literature. The results show that it effectively detects and separates the instabilities caused by problems from others. We also conduct an evaluation on the latest version of GNU Scientific Library, and the toolchain finds a few real bugs in the well-maintained and widely deployed numerical library. With the help of our toolchain, we report the details and fixing advices to the GSL buglist.
Author Enyi Tang
Zhenyu Chen
Xiangyu Zhang
Xuandong Li
Muller, Norbert T.
Author_xml – sequence: 1
  surname: Enyi Tang
  fullname: Enyi Tang
  email: eytang@nju.edu.cn
  organization: State Key Lab. for Novel Software Technol., Nanjing Univ., Nanjing, China
– sequence: 2
  surname: Xiangyu Zhang
  fullname: Xiangyu Zhang
  email: xyzhang@cs.purdue.edu
  organization: Dept. of Comput. Sci., Purdue Univ., West Lafayette, IN, USA
– sequence: 3
  givenname: Norbert T.
  surname: Muller
  fullname: Muller, Norbert T.
  email: mueller@uni-trier.de
  organization: Abt. Inf., Univ. of Trier, Trier, Germany
– sequence: 4
  surname: Zhenyu Chen
  fullname: Zhenyu Chen
  email: zychen@nju.edu.cn
  organization: State Key Lab. for Novel Software Technol., Nanjing Univ., Nanjing, China
– sequence: 5
  surname: Xuandong Li
  fullname: Xuandong Li
  email: lxd@nju.edu.cn
  organization: State Key Lab. for Novel Software Technol., Nanjing Univ., Nanjing, China
BookMark eNp9kE1LAzEQhoMo2FbvgpcFz1vzsdnsHKWtWhAVWs9LNs7WlDapSYr037u1xYMHT4HJ-8w7PH1y6rxDQq4YHTJG4XY-mww5ZeWQlwUHWZ6QHgMBuZCcnpIepVDlUlZwTvoxLimlUinZI-uZb9OXDpg9b9cYrNGrbOpi0o1d2bTLxpjQJOtdpt17NrZ64Xy0MWt22civG-usW2Sz5M2Hjsman9TUtd08Yf4a0Ni4h-fY_brFBTlr9Sri5fEdkLf7yXz0mD-9PExHd0-54cBSLnmjtBZCUNAgWmBUFciYUq1GUMBAS8qqxhSFatAgVIIhmEKAbCpQCsSA3Bz2boL_3Hbd9dJvg-sqa85UUfBKlLxL0UPKBB9jwLbeBLvWYVczWu-l1p3Uei-1PkrtkPIPYmzSez8paLv6D7w-gBYRf3u6W3kJhfgGMpmGmg
CODEN IESEDJ
CitedBy_id crossref_primary_10_1109_TC_2022_3224377
crossref_primary_10_1109_MCSE_2019_2947017
crossref_primary_10_1145_3715104
crossref_primary_10_1007_s11227_018_2552_x
crossref_primary_10_1145_3290369
Cites_doi 10.1145/1839480.1839486
10.1016/j.cpc.2008.02.003
10.1007/978-3-642-14203-1_11
10.1145/2048066.2048094
10.1145/2503210.2503296
10.1016/j.jlap.2005.07.002
10.1109/LICS.2004.1319637
10.1145/2509136.2509526
10.1109/FCCM.2006.21
10.1145/2594291.2594302
10.1145/1831708.1831724
10.1109/5992.852391
10.1007/3-540-47764-0_14
10.1145/2429069.2429133
10.1145/1353445.1353446
10.4324/9780203774441
10.1016/j.jlap.2004.07.003
10.1109/TSE.1976.233818
10.1137/1.9780898717716
10.1007/11823230_3
10.1002/spe.637
10.1109/ICST.2013.17
10.1007/s11786-011-0103-4
10.1145/355637.355639
10.1007/978-3-540-74061-2_19
10.1007/3-540-45927-8_14
10.1023/B:NUMA.0000049483.75679.ce
10.1016/S0129-6264(00)00021-4
10.1109/ARITH.2007.20
10.1145/2345156.2254118
10.1109/ARITH.2011.40
10.1145/1480945.1480960
10.1145/800125.804035
10.1137/1.9780898718027
10.1007/s11155-005-6891-y
10.1145/356502.356496
10.3354/cr030079
10.1017/S0960129506005822
10.1145/1236463.1236468
10.1016/0378-4754(93)90003-D
10.1109/ICSE.2015.70
10.1007/3-540-45335-0_14
10.1007/978-3-540-24738-8_18
10.1016/j.tcs.2005.09.058
10.1109/ICCD.2009.5413114
10.1145/1916461.1916469
10.51400/2709-6998.1958
10.1109/FPL.2007.4380629
10.1145/1134285.1134323
10.5540/tema.2003.04.03.0297
ContentType Journal Article
Copyright Copyright IEEE Computer Society 2017
Copyright_xml – notice: Copyright IEEE Computer Society 2017
DBID 97E
RIA
RIE
AAYXX
CITATION
JQ2
K9.
DOI 10.1109/TSE.2016.2642956
DatabaseName IEEE Xplore (IEEE)
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
ProQuest Computer Science Collection
ProQuest Health & Medical Complete (Alumni)
DatabaseTitle CrossRef
ProQuest Health & Medical Complete (Alumni)
ProQuest Computer Science Collection
DatabaseTitleList
ProQuest Health & Medical Complete (Alumni)
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 1939-3520
EndPage 994
ExternalDocumentID 10_1109_TSE_2016_2642956
7792694
Genre orig-research
GrantInformation_xml – fundername: NSF Award
  grantid: 1320306; 1320444; 1409668
– fundername: National Basic Research Program of China 973 Program
  grantid: 2014CB340702
– fundername: National Natural Science Foundation of China
  grantid: 61373013; 61402222; 61632015
  funderid: 10.13039/501100001809
GroupedDBID --Z
-DZ
-~X
.DC
0R~
29I
4.4
5GY
6IK
85S
8R4
8R5
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABPPZ
ABQJQ
ABVLG
ACGFO
ACGOD
ACIWK
ACNCT
AENEX
AGQYO
AHBIQ
AKJIK
AKQYR
ALMA_UNASSIGNED_HOLDINGS
ASUFR
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BKOMP
BPEOZ
CS3
DU5
EBS
EDO
EJD
HZ~
I-F
IEDLZ
IFIPE
IPLJI
JAVBF
LAI
M43
MS~
O9-
OCL
P2P
Q2X
RIA
RIE
RNS
RXW
S10
TAE
TN5
TWZ
UHB
UPT
WH7
YZZ
AAYXX
ALIPV
CITATION
RIG
JQ2
K9.
ID FETCH-LOGICAL-c291t-52b7aa33309a93f91074e1177fae97919a5018bc447bece9831e9c4395b897793
IEDL.DBID RIE
ISSN 0098-5589
IngestDate Mon Jun 30 11:09:17 EDT 2025
Thu Apr 24 23:00:37 EDT 2025
Tue Jul 01 01:53:15 EDT 2025
Wed Aug 27 02:47:44 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 10
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c291t-52b7aa33309a93f91074e1177fae97919a5018bc447bece9831e9c4395b897793
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0001-9004-1292
PQID 2174428362
PQPubID 21418
PageCount 20
ParticipantIDs crossref_citationtrail_10_1109_TSE_2016_2642956
crossref_primary_10_1109_TSE_2016_2642956
ieee_primary_7792694
proquest_journals_2174428362
ProviderPackageCode CITATION
AAYXX
PublicationCentury 2000
PublicationDate 2017-10-01
PublicationDateYYYYMMDD 2017-10-01
PublicationDate_xml – month: 10
  year: 2017
  text: 2017-10-01
  day: 01
PublicationDecade 2010
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE transactions on software engineering
PublicationTitleAbbrev TSE
PublicationYear 2017
Publisher IEEE
IEEE Computer Society
Publisher_xml – name: IEEE
– name: IEEE Computer Society
References ref57
ref56
stanoyevitch (ref19) 2011
ref15
ref58
ref14
ref53
ref52
ref55
ref54
ref10
ref16
cort (ref11) 2005; 30
chatelin (ref23) 1988; 16
ref51
ref50
ref46
ref45
ref48
liu (ref17) 2009; 17
ref42
ref41
xiao (ref18) 0; 1
ref44
ref43
ref49
cohen (ref13) 2013
ref8
ref7
granlund (ref59) 2014
ref9
galassi (ref26) 2002
ref4
ref3
ref40
ref35
ref34
ref37
ref36
ref31
parker (ref25) 1997
ref30
ref33
vignes (ref63) 0
ref2
ref1
ref39
ref38
cheney (ref12) 2007
ref24
wilkinson (ref32) 1994
ref64
ref22
ref21
ref28
ref27
parker (ref20) 0
ref29
mohan (ref5) 2002
lam (ref47) 0
ref60
ref62
shoup (ref61) 2014
hart (ref6) 2010
References_xml – ident: ref57
  doi: 10.1145/1839480.1839486
– ident: ref1
  doi: 10.1016/j.cpc.2008.02.003
– ident: ref43
  doi: 10.1007/978-3-642-14203-1_11
– ident: ref44
  doi: 10.1145/2048066.2048094
– ident: ref48
  doi: 10.1145/2503210.2503296
– year: 2002
  ident: ref26
  publication-title: GNU Scientific Library Reference Manual
– ident: ref53
  doi: 10.1016/j.jlap.2005.07.002
– ident: ref51
  doi: 10.1109/LICS.2004.1319637
– ident: ref38
  doi: 10.1145/2509136.2509526
– year: 1994
  ident: ref32
  publication-title: Rounding Errors in Algebraic Processes
– ident: ref56
  doi: 10.1109/FCCM.2006.21
– ident: ref49
  doi: 10.1145/2594291.2594302
– start-page: 610
  year: 0
  ident: ref63
  article-title: Error analysis in computing
  publication-title: Proc Int Federation Inf Process
– ident: ref14
  doi: 10.1145/1831708.1831724
– ident: ref64
  doi: 10.1109/5992.852391
– ident: ref36
  doi: 10.1007/3-540-47764-0_14
– ident: ref4
  doi: 10.1145/2429069.2429133
– ident: ref46
  doi: 10.1145/1353445.1353446
– year: 2013
  ident: ref13
  publication-title: Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences
  doi: 10.4324/9780203774441
– year: 0
  ident: ref20
  article-title: Monte Carlo arithmetic: A framework for the statistical analysis of roundoff error
– ident: ref52
  doi: 10.1016/j.jlap.2004.07.003
– ident: ref39
  doi: 10.1109/TSE.1976.233818
– ident: ref33
  doi: 10.1137/1.9780898717716
– ident: ref21
  doi: 10.1007/11823230_3
– year: 2007
  ident: ref12
  article-title: Numerical mathematics and computing
– ident: ref24
  doi: 10.1002/spe.637
– ident: ref40
  doi: 10.1109/ICST.2013.17
– ident: ref16
  doi: 10.1007/s11786-011-0103-4
– ident: ref30
  doi: 10.1145/355637.355639
– ident: ref2
  doi: 10.1007/978-3-540-74061-2_19
– ident: ref45
  doi: 10.1007/3-540-45927-8_14
– ident: ref8
  doi: 10.1023/B:NUMA.0000049483.75679.ce
– ident: ref15
  doi: 10.1016/S0129-6264(00)00021-4
– ident: ref41
  doi: 10.1109/ARITH.2007.20
– ident: ref37
  doi: 10.1145/2345156.2254118
– year: 2014
  ident: ref59
  article-title: The GMP development team
  publication-title: GNU MP The GNU Multiple Precision Arithmetic Library
– ident: ref42
  doi: 10.1109/ARITH.2011.40
– ident: ref3
  doi: 10.1145/1480945.1480960
– ident: ref29
  doi: 10.1145/800125.804035
– ident: ref28
  doi: 10.1137/1.9780898718027
– start-page: 369
  year: 0
  ident: ref47
  article-title: Automatically adapting programs for mixed-precision floating-point computation
  publication-title: Proc 27th Int ACM Conf Int Conf Supercomputing
– year: 2011
  ident: ref19
  publication-title: Introduction to Numerical Ordinary and Partial Differential Equations Using MATLAB
– ident: ref54
  doi: 10.1007/s11155-005-6891-y
– ident: ref31
  doi: 10.1145/356502.356496
– volume: 30
  start-page: 79
  year: 2005
  ident: ref11
  article-title: Advantages of the mean absolute error mae over the root mean square error RMSE in assessing average model performance
  publication-title: Climate Res
  doi: 10.3354/cr030079
– ident: ref62
  doi: 10.1017/S0960129506005822
– ident: ref9
  doi: 10.1145/1236463.1236468
– year: 2010
  ident: ref6
  publication-title: Power Electronics
– volume: 16
  start-page: 851
  year: 1988
  ident: ref23
  article-title: Sur le taux de fiabilité général de la méthode CESTAC
  publication-title: Comptes Rendus de l'Académie des Sciences - Series IV - Physics
– ident: ref7
  doi: 10.1016/0378-4754(93)90003-D
– ident: ref27
  doi: 10.1109/ICSE.2015.70
– ident: ref10
  doi: 10.1007/3-540-45335-0_14
– year: 2002
  ident: ref5
  publication-title: POWER ELECTRONICS Converters Applications and Design
– year: 2014
  ident: ref61
– ident: ref35
  doi: 10.1007/978-3-540-24738-8_18
– volume: 1
  start-page: 622
  year: 0
  ident: ref18
  article-title: Research on ill-conditioned problem and modeling precision in GM(1,1) model
  publication-title: Proc IEEE Int Conf Syst Man Cybern
– ident: ref50
  doi: 10.1016/j.tcs.2005.09.058
– ident: ref55
  doi: 10.1109/ICCD.2009.5413114
– ident: ref60
  doi: 10.1145/1916461.1916469
– volume: 17
  start-page: 216
  year: 2009
  ident: ref17
  article-title: Novel methods for solving severely ill-posed linear equations system
  publication-title: J Marine Sci Technol
  doi: 10.51400/2709-6998.1958
– ident: ref58
  doi: 10.1109/FPL.2007.4380629
– ident: ref22
  doi: 10.1145/1134285.1134323
– ident: ref34
  doi: 10.5540/tema.2003.04.03.0297
– year: 1997
  ident: ref25
  article-title: Monte Carlo arithmetic: Exploiting randomness in floating-point arithmetic
SSID ssj0005775
ssib053395008
Score 2.30584
Snippet Numerical instability is a well-known problem that may cause serious runtime failures. This paper discusses the reason of instability in software development...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 975
SubjectTerms Algorithm design and analysis
Computer bugs
infinite-precision arithmetic
Libraries
Numerical analysis
Software
Software algorithms
Software development
Software testing
Stability
stochastic arithmetic
Title Software Numerical Instability Detection and Diagnosis by Combining Stochastic and Infinite-Precision Testing
URI https://ieeexplore.ieee.org/document/7792694
https://www.proquest.com/docview/2174428362
Volume 43
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1NSwMxEB20Jy9-i9UqOXgR3HY_m52jqKUKFaEteFuSbFbFupW6RfTXm8nuVlERb3tIQmAmmcnOm_cAjoSvUi_jroOBDJxQKNeJYykcky2nqQjjKBXUOzy47vbH4dVtdLsEJ4teGK21BZ_pNn3aWn46VXP6VdbhHKnxchmWjZuVvVqfcA7Oo5ofM4pirEuSLnZGwwvCcHXbJvj7SFLVX0KQ1VT5cRHb6NJbg0G9rxJU8tieF7Kt3r9RNv534-uwWqWZ7LT0iw1Y0vkmrNUSDqw60VvwNDTX8KuYaXY9L2s3E0b4gZK9-42d68JitXIm8pSdl7i8hxcm35hZTFp1CTYspupeEOGzHXWZZw-UyTo3s0rAh42IyyO_24Zx72J01ncqBQZH-egV5pUquRBBELgoMMiQ0JuayryZ0MjRQ0F8gFKFITe-oDEOPI3K5DiRjE1iicEONPJprneBKYzQlT6a2KDCVHOUbqajMFQi84Sn_SZ0aqMkqqInJ5WMSWKfKS4mxowJmTGpzNiE48WM55Ka44-xW2SVxbjKIE1o1XZPqrP7ktAjjWjouv7e77P2YcWn4G4hfS1oFLO5PjCpSSEPrU9-ANI_4Vs
linkProvider IEEE
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LT9wwEB5ROLSXAqUVS3n40EulZjcPe505Ih5aHruqtIvELbIdBxCQrZasEP31eJxkqWhVccvBiSx9tmec-eb7AL6p2ORRIcMAE50EXJkwSFOtApct57niqcgV9Q4PR_3BBT-9FJdL8GPRC2Ot9eQz26VHX8vPp2ZOv8p6UiI1Xr6DFRf3uai7tV4IHVKKViFTiBTbomSIvcn4iFhc_a4L_zGSWfUfQci7qvx1FPv4crwKw3ZmNa3ktjuvdNf8fiXa-Napr8HHJtFk-_XKWIclW36C1dbEgTV7egPux-4gflQzy0bzunpzx4hBUOt3P7FDW3m2VslUmbPDmpl388D0E3Mf095fgo2rqblWJPnsR52UxQ3lssHPWWPhwyak5lFefYaL46PJwSBoPBgCE2NUuXuqlkolSRKiwqRA4m9aKvQWyqLECBUpAmrDuXSrwWKaRBaNy3KETl1qickXWC6npd0EZlBgqGN00cHw3ErUYWEF50YVkYps3IFeC0pmGoFy8sm4y_xFJcTMwZgRjFkDYwe-L974VYtz_GfsBqGyGNcA0oHtFves2b0PGV3TSIiuH2_9-609eD-YDM-z85PR2Vf4EFOo9wS_bViuZnO74xKVSu_69fkMxUzkqA
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Software+Numerical+Instability+Detection+and+Diagnosis+by+Combining+Stochastic+and+Infinite-Precision+Testing&rft.jtitle=IEEE+transactions+on+software+engineering&rft.au=Tang%2C+Enyi&rft.au=Zhang%2C+Xiangyu&rft.au=Muller%2C+Norbert+T&rft.au=Chen%2C+Zhenyu&rft.date=2017-10-01&rft.pub=IEEE+Computer+Society&rft.issn=0098-5589&rft.eissn=1939-3520&rft.volume=43&rft.issue=10&rft.spage=975&rft_id=info:doi/10.1109%2FTSE.2016.2642956&rft.externalDBID=NO_FULL_TEXT
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0098-5589&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0098-5589&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0098-5589&client=summon