Systematic literature review on software quality for AI-based software
There is a widespread demand for Artificial Intelligence (AI) software, specifically Machine Learning (ML). It is getting increasingly popular and being adopted in various applications we use daily. AI-based software quality is different from traditional software quality because it generally address...
Saved in:
| Published in | Empirical software engineering : an international journal Vol. 27; no. 3 |
|---|---|
| Main Authors | , |
| Format | Journal Article |
| Language | English |
| Published |
New York
Springer US
01.05.2022
Springer Nature B.V |
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | There is a widespread demand for Artificial Intelligence (AI) software, specifically Machine Learning (ML). It is getting increasingly popular and being adopted in various applications we use daily. AI-based software quality is different from traditional software quality because it generally addresses distinct and more complex kinds of problems. With the fast advance of AI technologies and related techniques, how to build high-quality AI-based software becomes a very prominent subject. This paper aims at investigating the state of the art on software quality (SQ) for AI-based systems and identifying quality attributes, applied models, challenges, and practices that are reported in the literature. We carried out a systematic literature review (SLR) from 1988 to 2020 to (i) analyze and understand related primary studies and (ii) synthesize limitations and open challenges to drive future research. Our study provides a road map for researchers to understand quality challenges, attributes, and practices in the context of software quality for AI-based software better. From the empirical evidence that we have gathered by this SLR, we suggest future work on this topic be structured under three categories which are Definition/Specification, Design/Evaluation, and Process/Socio-technical. |
|---|---|
| AbstractList | There is a widespread demand for Artificial Intelligence (AI) software, specifically Machine Learning (ML). It is getting increasingly popular and being adopted in various applications we use daily. AI-based software quality is different from traditional software quality because it generally addresses distinct and more complex kinds of problems. With the fast advance of AI technologies and related techniques, how to build high-quality AI-based software becomes a very prominent subject. This paper aims at investigating the state of the art on software quality (SQ) for AI-based systems and identifying quality attributes, applied models, challenges, and practices that are reported in the literature. We carried out a systematic literature review (SLR) from 1988 to 2020 to (i) analyze and understand related primary studies and (ii) synthesize limitations and open challenges to drive future research. Our study provides a road map for researchers to understand quality challenges, attributes, and practices in the context of software quality for AI-based software better. From the empirical evidence that we have gathered by this SLR, we suggest future work on this topic be structured under three categories which are Definition/Specification, Design/Evaluation, and Process/Socio-technical. |
| ArticleNumber | 66 |
| Author | Tarhan, Ayça Kolukısa Gezici, Bahar |
| Author_xml | – sequence: 1 givenname: Bahar orcidid: 0000-0001-6704-3134 surname: Gezici fullname: Gezici, Bahar email: bahargezici@cs.hacettepe.edu.tr organization: Institute of Science, Computer Engineering Department, Hacettepe University – sequence: 2 givenname: Ayça Kolukısa surname: Tarhan fullname: Tarhan, Ayça Kolukısa organization: Institute of Science, Computer Engineering Department, Hacettepe University |
| BookMark | eNp9kE1LAzEQhoNUsK3-AU8LnqOZfO3usRSrBcGDeg7ZbCJb2k2bpJb-e6MrCh56mmHmfebjnaBR73uL0DWQWyCkvItApOSYUMBAgAhMz9AYRMlwKUGOcs4qihkV8gJNYlwRQuqSizFavBxjshudOlOsu2SDTvtgi2A_OnsofF9E79JB59Jur7PgWDgfitkSNzra9rd7ic6dXkd79ROn6G1x_zp_xE_PD8v57AkbVomEZdPUpKWOC1Ea7biDkkpmtaUAriHGtgBlRXhdVZLThraC12AqqjUT1DnDpuhmmLsNfre3MamV34c-r1RUciIkk1RkVTWoTPAxBuuU6VJ-0fcp6G6tgKgv19TgmsquqW_XFM0o_YduQ7fR4XgaYgMUs7h_t-HvqhPUJw2LgQE |
| CitedBy_id | crossref_primary_10_1109_ACCESS_2023_3317798 crossref_primary_10_1145_3698111 crossref_primary_10_1109_MC_2023_3240730 crossref_primary_10_1016_j_infsof_2025_107678 crossref_primary_10_3390_su16145901 crossref_primary_10_3390_app12178700 crossref_primary_10_1007_s10515_024_00453_w crossref_primary_10_1109_TSE_2023_3308952 crossref_primary_10_1145_3716497 crossref_primary_10_3233_AIS_220483 crossref_primary_10_1016_j_jss_2024_112151 |
| Cites_doi | 10.1109/ICTKE.2017.8259629 10.1109/ICSME.2019.00051 10.1109/ICSTW.2018.00061 10.1016/S0065-2458(05)65001-2 10.4018/978-1-7998-5101-1.ch001 10.1007/978-3-030-58793-2_2 10.1016/j.aei.2006.11.006 10.1109/ICSTW.2018.00060 10.18293/SEKE2017-176 10.1109/CESSER-IP.2019.00009 10.3923/itj.2007.835.842 10.1109/TELFOR.2016.7818902 10.1109/ISSREW.2019.00035 10.1016/j.infsof.2020.106368 10.1080/14783363.2017.1310703 10.1109/MSP.2017.2762725 10.1109/ACCESS.2019.2937107 10.1145/3338906.3338937 10.1007/s10664-020-09881-0 10.1145/3371158.3371233 10.1109/GCCE.2018.8574766 10.1109/69.368522 10.1109/RE48521.2020.00036 10.1007/978-3-030-58793-2_5 10.1109/REW.2019.00050 10.1145/3368089.3417039 10.1016/j.jss.2020.110542 10.1145/2601248.2601268 10.1109/SEAA.2018.00018 10.1109/ICDEW.2007.4401084 10.1007/978-3-030-65854-0_4 10.1049/sfw2.12011 10.2991/jase.d.190131.001 10.1109/EIT48999.2020.9208288 10.1080/09720529.2020.1721883 10.1007/978-3-642-29044-2 10.1007/s10994-020-05872-w 10.1145/1463788.1463807 10.1145/3487043 10.1145/2915970.2916008 10.1007/978-3-662-43839-8 10.1016/j.infsof.2015.03.007 10.1007/978-3-030-32409-4_1 10.1007/978-3-030-41418-4_3 10.1109/QRS51102.2020.00016 10.1109/BigDataCongress.2018.00029 10.1109/TSE.2019.2937083 10.1145/3194085.3194090 10.1109/RE.2019.00050 10.1007/s10664-010-9146-4 |
| ContentType | Journal Article |
| Copyright | The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022 The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022. |
| Copyright_xml | – notice: The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022 – notice: The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022. |
| DBID | AAYXX CITATION 7SC 8FD 8FE 8FG ABJCF AFKRA ARAPS BENPR BGLVJ CCPQU DWQXO HCIFZ JQ2 L6V L7M L~C L~D M7S P5Z P62 PHGZM PHGZT PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PTHSS S0W |
| DOI | 10.1007/s10664-021-10105-2 |
| DatabaseName | CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central UK/Ireland Advanced Technologies & Aerospace Collection ProQuest Central (New) Technology Collection ProQuest One ProQuest Central Korea SciTech Premium Collection ProQuest Computer Science Collection ProQuest Engineering Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional Engineering Database ProQuest advanced technologies & aerospace journals ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Premium ProQuest One Academic (New) ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection DELNET Engineering & Technology Collection |
| DatabaseTitle | CrossRef Technology Collection Technology Research Database Computer and Information Systems Abstracts – Academic ProQuest One Academic Middle East (New) ProQuest Advanced Technologies & Aerospace Collection ProQuest Computer Science Collection Computer and Information Systems Abstracts SciTech Premium Collection ProQuest One Community College ProQuest Central China ProQuest Central ProQuest One Applied & Life Sciences ProQuest Engineering Collection ProQuest Central Korea ProQuest Central (New) Advanced Technologies Database with Aerospace Engineering Collection Advanced Technologies & Aerospace Collection Engineering Database ProQuest One Academic Eastern Edition ProQuest Technology Collection ProQuest SciTech Collection Computer and Information Systems Abstracts Professional Advanced Technologies & Aerospace Database ProQuest One Academic UKI Edition ProQuest DELNET Engineering and Technology Collection Materials Science & Engineering Collection ProQuest One Academic ProQuest One Academic (New) |
| DatabaseTitleList | Technology Collection |
| Database_xml | – sequence: 1 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 1573-7616 |
| ExternalDocumentID | 10_1007_s10664_021_10105_2 |
| GroupedDBID | -4Z -59 -5G -BR -EM -Y2 -~C .86 .DC .VR 06D 0R~ 0VY 199 1N0 1SB 2.D 203 28- 29G 2J2 2JN 2JY 2KG 2LR 2P1 2VQ 2~H 30V 4.4 406 408 409 40D 40E 5GY 5QI 5VS 67Z 6NX 78A 8FE 8FG 8TC 8UJ 95- 95. 95~ 96X AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AAOBN AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYOK AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDZT ABECU ABFTD ABFTV ABHLI ABHQN ABJCF ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTEG ABTHY ABTKH ABTMW ABULA ABWNU ABXPI ACAOD ACBXY ACDTI ACGFS ACHSB ACHXU ACIWK ACKNC ACMDZ ACMLO ACOKC ACOMO ACPIV ACSNA ACZOJ ADHHG ADHIR ADIMF ADINQ ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF ADZKW AEBTG AEFIE AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFEXP AFGCZ AFKRA AFLOW AFQWF AFWTZ AFZKB AGAYW AGDGC AGGDS AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMXSW AMYLF AMYQR AOCGG ARAPS ARMRJ ASPBG AVWKF AXYYD AYJHY AZFZN B-. BA0 BBWZM BDATZ BENPR BGLVJ BGNMA BSONS CAG CCPQU COF CS3 CSCUP DDRTE DL5 DNIVK DPUIP DU5 EBLON EBS EIOEI EJD ESBYG FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNWQR GQ6 GQ7 GQ8 GXS H13 HCIFZ HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ I09 IHE IJ- IKXTQ ITM IWAJR IXC IZIGR IZQ I~X I~Z J-C J0Z JBSCW JCJTX JZLTJ KDC KOV KOW L6V LAK LLZTM M4Y M7S MA- N2Q NB0 NDZJH NPVJJ NQJWS NU0 O9- O93 O9G O9I O9J OAM P19 P62 P9O PF0 PT4 PT5 PTHSS Q2X QOK QOS R4E R89 R9I RHV RNI RNS ROL RPX RSV RZC RZE RZK S0W S16 S1Z S26 S27 S28 S3B SAP SCJ SCLPG SCO SDH SDM SHX SISQX SJYHP SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 T16 TSG TSK TSV TUC U2A UG4 UOJIU UTJUX UZXMN VC2 VFIZW W23 W48 WK8 YLTOR Z45 Z7R Z7S Z7V Z7X Z7Z Z81 Z83 Z86 Z88 Z8M Z8N Z8P Z8R Z8T Z8U Z8W Z92 ZMTXR ~EX AAPKM AAYXX ABBRH ABDBE ABFSG ACSTC ADHKG AEZWR AFDZB AFHIU AFOHR AGQPQ AHPBZ AHWEU AIXLP ATHPR AYFIA CITATION PHGZM PHGZT 7SC 8FD ABRTQ DWQXO JQ2 L7M L~C L~D PKEHL PQEST PQGLB PQQKQ PQUKI PRINS |
| ID | FETCH-LOGICAL-c385t-6bb90d2f4557caf4f17263eae211fb0ced117804988642b2d5491c82aa352ffc3 |
| IEDL.DBID | U2A |
| ISSN | 1382-3256 |
| IngestDate | Fri Jul 25 12:21:50 EDT 2025 Tue Jul 01 03:32:20 EDT 2025 Thu Apr 24 23:01:03 EDT 2025 Fri Feb 21 02:46:06 EST 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Keywords | Measurement Machine learning Software quality Product quality model Quality attributes Artificial intelligence Quality metrics |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c385t-6bb90d2f4557caf4f17263eae211fb0ced117804988642b2d5491c82aa352ffc3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0001-6704-3134 |
| PQID | 2640563625 |
| PQPubID | 326341 |
| ParticipantIDs | proquest_journals_2640563625 crossref_citationtrail_10_1007_s10664_021_10105_2 crossref_primary_10_1007_s10664_021_10105_2 springer_journals_10_1007_s10664_021_10105_2 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2022-05-01 |
| PublicationDateYYYYMMDD | 2022-05-01 |
| PublicationDate_xml | – month: 05 year: 2022 text: 2022-05-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York – name: Dordrecht |
| PublicationSubtitle | An International Journal |
| PublicationTitle | Empirical software engineering : an international journal |
| PublicationTitleAbbrev | Empir Software Eng |
| PublicationYear | 2022 |
| Publisher | Springer US Springer Nature B.V |
| Publisher_xml | – name: Springer US – name: Springer Nature B.V |
| References | Rushby J (1988) Quality measures and assurance for AI software, vol 18. National Aeronautics and Space Administration, Scientific and Technical Information Division Henriksson J, Borg M, Englund C (2018) Automotive safety and machine learning: Initial results from a study on how to adapt the iso 26262 safety standard. In: 2018 IEEE/ACM 1St international workshop on software engineering for AI in autonomous systems (SEFAIAS). IEEE, pp 47–49 HopgoodAAThe state of artificial intelligenceAdv Comput20056517510.1016/S0065-2458(05)65001-2 Masuda S, Ono K, Yasue T, Hosokawa N (2018) A survey of software quality for machine learning applications. In: 2018 IEEE International conference on software testing, verification and validation workshops (ICSTW). IEEE, pp 279–284 PetersenKVakkalankaSKuzniarzLGuidelines for conducting systematic mapping studies in software engineering: an updateInf Softw Technol20156411810.1016/j.infsof.2015.03.007 Aggarwal A, Lohia P, Nagar S, Dey K, Saha D (2019) Black box fairness testing of machine learning models. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp 625–635 ChenRBastaniFBTsaoTWOn the reliability of ai planning software in real-time applicationsIEEE Trans Knowl Data Eng19957141310.1109/69.368522 Kuwajima H, Ishikawa F (2019) Adapting square for quality assessment of artificial intelligence systems. In: 2019 IEEE International symposium on software reliability engineering workshops (ISSREW). IEEE, pp 13–18 Wan Z, Xia X, Lo D, Murphy GC (2019) How does machine learning change software development practices? IEEE Transactions on Software Engineering 25012:2008 I (2008) software engineering — software product quality requirements and evaluation (square) — data quality model. https://www.iso.org/standard/35736.html Geske F, Hofmann P, Lämmermann L, Schlatt V, Urbach N (2021) Gateways to artificial intelligence: Developing a taxonomy for ai service platforms. In: Twenty-ninth european conference on information systems (ECIS) 29119-1:2013 I (2013) Software and systems engineering — software testing. https://www.iso.org/standard/45142.html Lakshen GA, Vraneš S., Janev V (2016) Big data and quality: A literature review. In: 2016 24Th telecommunications forum (TELFOR). IEEE, pp 1–4 Tao C, Gao J, Wang T (2019) Testing and quality validation for ai software–perspectives, issues, and practices. IEEE Access 7:120164–120175 Turhan B, Kutlubay O (2007) Mining software data. In: 2007 IEEE 23Rd international conference on data engineering workshop. IEEE, pp 912–916 KitchenhamBProcedures for performing systematic reviews. keele, UKKeele Univ2004332004126 Arpteg A, Brinne B, Crnkovic-Friis L, Bosch J (2018) Software engineering challenges of deep learning. In: 2018 44Th euromicro conference on software engineering and advanced applications (SEAA). IEEE, pp 50–59 9126-1:2001 I (2001) Software engineering — product quality. https://www.iso.org/standard/22749.html 25000 I (2005) The iso/iec 25000 series of standards. https://iso25000.com/index.php/en/iso-25000-standards Mannarswamy S, Roy S, Chidambaram S (2020) Tutorial on software testing & quality assurance for machine learning applications from research bench to real world. In: Proceedings of the 7th ACM IKDD CoDS and 25th COMAD, pp 373–374 Russel S, Norvig P (2009) Artificial intelligence: a modern approach, English Lenarduzzi V, Lomio F, Moreschini S, Taibi D, Tamburri DA (2021) Software quality for ai: Where we are now?. In: International conference on software quality. Springer, pp 43–53 26262-1:2018 I (2018) Road vehicles — functional safety. https://www.iso.org/standard/68383.html ISO/IECIso/iec 25010 (2011)-systems and software quality requirements and evaluation (square)-system and software quality modelsInternational Standard ISO/IEC 25010201121125 Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering - Garousi V, Felderer M, Mäntylä MV (2016) The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In: Proceedings of the 20th international conference on evaluation and assessment in software engineering, pp 1–6 Byrne C (2017) Development Workflows for Data Scientists. O’Reilly Media KuwajimaHYasuokaHNakaeTEngineering problems in machine learning systemsMach Learn2020109511031126410156810.1007/s10994-020-05872-w07224993 Taleb I, Serhani MA, Dssouli R (2018) Big data quality: a survey. In: 2018 IEEE International congress on big data (bigdata congress). IEEE, pp 166–173 Pons L, Ozkaya I (2019) Priority quality attributes for engineering ai-enabled systems. arXiv:1911.02912 Bosch J, Olsson HH, Crnkovic I (2021) Engineering ai systems: a research agenda. In: Artificial intelligence paradigms for smart cyber-physical systems. IGI Global, pp 1–19 IvarssonMGorschekTA method for evaluating rigor and industrial relevance of technology evaluationsEmpir Softw Eng201116336539510.1007/s10664-010-9146-4 Murphy C, Kaiser GE, Arias M (2006) A framework for quality assurance of machine learning applications - RiccioVJahangirovaGStoccoAHumbatovaNWeissMTonellaPTesting machine learning based systems: a systematic mappingEmpir Softw Eng20202565193525410.1007/s10664-020-09881-0 Nakajima S (2019) Distortion and faults in machine learning software. In: International workshop on structured object-oriented formal language and method. Springer, pp 29–41 Ongsulee P (2017) Artificial intelligence, machine learning and deep learning. In: 2017 15Th international conference on ICT and knowledge engineering (ICT&KE). IEEE, pp 1–6 DengLArtificial intelligence in the rising wave of deep learning: The historical path and future outlook [perspectives]IEEE Signal Proc Mag201835118017710.1109/MSP.2017.2762725 Hamada K, Ishikawa F, Masuda S, Matsuya M, Ujita Y (2020) Guidelines for quality assurance of machine learning-based artificial intelligence. In: SEKE2020: The 32nd international conference on software engineering & knowledge engineering, pp 335–341 Borg M, Englund C, Wnuk K, Duran B, Levandowski C, Gao S, Tan Y, Kaijser H, Lönn H, Törnqvist J (2018) Safely entering the deep: A review of verification and validation for machine learning and a challenge elicitation in the automotive industry. arXiv:1812.05389 Forward A, Lethbridge TC (2008) A taxonomy of software types to facilitate search and evidence-based software engineering. In: Proceedings of the 2008 conference of the center for advanced studies on collaborative research: meeting of minds, pp 179–191 Hyun ParkSSeon ShinWHyun ParkYLeeYBuilding a new culture for quality management in the era of the fourth industrial revolutionTotal Qual Manag Bus Excell2017289-1093494510.1080/14783363.2017.1310703 de Almeida BiolchiniJCMianPGNataliACCConteTTravassosGHScientific research ontology to support systematic review in software engineeringAdv Eng Inform200721213315110.1016/j.aei.2006.11.006 Vogelsang A, Borg M (2019) Requirements engineering for machine learning: Perspectives from data scientists. In: 2019 IEEE 27Th international requirements engineering conference workshops (REW). IEEE, pp 245–251 Zhang JM, Harman M, Ma L, Liu Y (2020) Machine learning testing: survey, landscapes and horizons. IEEE Transactions on Software Engineering Nakajima S (2018) Quality assurance of machine learning software. In: 2018 IEEE 7Th global conference on consumer electronics (GCCE). IEEE, pp 601–604 MalikVSinghSArtificial intelligent environments: risk management and quality assurance implementationJ Discret Math Sci Cryptogr202023118719510.1080/09720529.2020.172188307477051 Tao C, Hao C, Gao J, Wang T, Wen W (2017) A practical study on quality evaluation for age recognition systems. In: SEKE, pp 345–350 Horkoff J (2019) Non-functional requirements for machine learning: Challenges and new directions. In: 2019 IEEE 27Th international requirements engineering conference (RE). IEEE, pp 386–391 VinayagasundaramBSrivatsaSSoftware quality in artificial intelligence systemInf Technol J20076683584210.3923/itj.2007.835.842 Zhang P, Cao W, Muccini H (2020) Quality assurance technologies of big data applications: A systematic literature review. arXiv:2002.01759 Siebert J, Joeckel L, Heidrich J, Nakamichi K, Ohashi K, Namba I, Yamamoto R, Aoyama M (2020) Towards guidelines for assessing qualities of machine learning systems. In: International conference on the quality of information and communications technology. Springer, pp 17–31 Martínez-Fernández S, Bogner J, Franch X, Oriol M, Siebert J, Trendowicz A, Vollmer AM, Wagner S (2021) Software engineering for ai-based systems: A survey. arXiv:2105.01984 BraiekHBKhomhFOn testing machine learning programsJ Syst Softw202016411054210.1016/j.jss.2020.110542 Poth A, Meyer B, Schlicht P, Riel A (2020) Quality assurance for machine learning–an approach to function and system safeguarding. In: 2020 IEEE 20Th international conference on software quality, reliability and security (QRS). IEEE, pp 22–29 HannousseASearching relevant papers for software engineering secondary studies: Semantic scholar coverage and identification roleIET Softw202115112614610.1049/sfw2.12011 Rahman MS, Reza H (2020) Systematic mapping study of non-functional requirements in big data system. In: 2020 IEEE International conference on electro information technology (EIT). IEEE, pp 025–031 Bourque P, Dupuis R, Abran A, Moore JW, Tripp L (2004) Guide to the software engineering body of knowledge - Alamin MAA, Uddin G (2021) Quality assurance challenges for machine learning software applications during software development life cycle phases. arXiv:2105.01195 Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer Science & Business Media Nascimento E, Nguyen-Duc A, Sundbø I, Conte T (2020) Software engineering for artificial intelligence and machine learning software: A systematic literature review. arXiv:2 10105_CR7 10105_CR8 10105_CR44 10105_CR43 10105_CR42 10105_CR48 HB Braiek (10105_CR14) 2020; 164 10105_CR47 10105_CR46 10105_CR45 10105_CR49 B Kitchenham (10105_CR32) 2004; 33 V Riccio (10105_CR57) 2020; 25 10105_CR73 10105_CR72 10105_CR71 10105_CR70 10105_CR33 10105_CR74 10105_CR37 10105_CR35 10105_CR34 V Malik (10105_CR41) 2020; 23 LE Lwakatare (10105_CR40) 2020; 127 10105_CR39 L Deng (10105_CR18) 2018; 35 10105_CR38 10105_CR1 10105_CR2 10105_CR5 10105_CR6 10105_CR3 R Chen (10105_CR16) 1995; 7 AA Hopgood (10105_CR26) 2005; 65 10105_CR4 A Hannousse (10105_CR24) 2021; 15 H Kuwajima (10105_CR36) 2020; 109 10105_CR62 10105_CR61 10105_CR60 10105_CR22 10105_CR66 10105_CR21 10105_CR65 10105_CR20 10105_CR64 10105_CR63 10105_CR25 10105_CR69 K Petersen (10105_CR53) 2015; 64 10105_CR68 10105_CR23 10105_CR29 10105_CR27 S Hyun Park (10105_CR28) 2017; 28 JC de Almeida Biolchini (10105_CR9) 2007; 21 ISO/IEC (10105_CR30) 2011; 2 10105_CR51 10105_CR50 10105_CR11 M Ivarsson (10105_CR31) 2011; 16 10105_CR55 10105_CR10 10105_CR54 10105_CR52 10105_CR15 10105_CR59 10105_CR58 10105_CR13 10105_CR12 10105_CR56 10105_CR19 10105_CR17 B Vinayagasundaram (10105_CR67) 2007; 6 |
| References_xml | – reference: Garousi V, Felderer M, Mäntylä MV (2016) The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In: Proceedings of the 20th international conference on evaluation and assessment in software engineering, pp 1–6 – reference: Geske F, Hofmann P, Lämmermann L, Schlatt V, Urbach N (2021) Gateways to artificial intelligence: Developing a taxonomy for ai service platforms. In: Twenty-ninth european conference on information systems (ECIS) – reference: VinayagasundaramBSrivatsaSSoftware quality in artificial intelligence systemInf Technol J20076683584210.3923/itj.2007.835.842 – reference: Ongsulee P (2017) Artificial intelligence, machine learning and deep learning. In: 2017 15Th international conference on ICT and knowledge engineering (ICT&KE). IEEE, pp 1–6 – reference: Borg M, Englund C, Wnuk K, Duran B, Levandowski C, Gao S, Tan Y, Kaijser H, Lönn H, Törnqvist J (2018) Safely entering the deep: A review of verification and validation for machine learning and a challenge elicitation in the automotive industry. arXiv:1812.05389 – reference: Mannarswamy S, Roy S, Chidambaram S (2020) Tutorial on software testing & quality assurance for machine learning applications from research bench to real world. In: Proceedings of the 7th ACM IKDD CoDS and 25th COMAD, pp 373–374 – reference: Rahman MS, Reza H (2020) Systematic mapping study of non-functional requirements in big data system. In: 2020 IEEE International conference on electro information technology (EIT). IEEE, pp 025–031 – reference: Siebert J, Joeckel L, Heidrich J, Nakamichi K, Ohashi K, Namba I, Yamamoto R, Aoyama M (2020) Towards guidelines for assessing qualities of machine learning systems. In: International conference on the quality of information and communications technology. Springer, pp 17–31 – reference: Nascimento E, Nguyen-Duc A, Sundbø I, Conte T (2020) Software engineering for artificial intelligence and machine learning software: A systematic literature review. arXiv:2011.03751 – reference: Vogelsang A, Borg M (2019) Requirements engineering for machine learning: Perspectives from data scientists. In: 2019 IEEE 27Th international requirements engineering conference workshops (REW). IEEE, pp 245–251 – reference: Wieringa RJ (2014) Design science methodology for information systems and software engineering. Springer – reference: Nishi Y, Masuda S, Ogawa H, Uetsuki K (2018) A test architecture for machine learning product. In: 2018 IEEE International conference on software testing, verification and validation workshops (ICSTW). IEEE, pp 273–278 – reference: Zhang JM, Harman M, Ma L, Liu Y (2020) Machine learning testing: survey, landscapes and horizons. IEEE Transactions on Software Engineering – reference: HopgoodAAThe state of artificial intelligenceAdv Comput20056517510.1016/S0065-2458(05)65001-2 – reference: Cummaudo A, Vasa R, Grundy J, Abdelrazek M, Cain A (2019) Losing confidence in quality: Unspoken evolution of computer vision services. In: 2019 IEEE International conference on software maintenance and evolution (ICSME). IEEE, pp 333–342 – reference: 25000 I (2005) The iso/iec 25000 series of standards. https://iso25000.com/index.php/en/iso-25000-standards – reference: Nakajima S (2018) Quality assurance of machine learning software. In: 2018 IEEE 7Th global conference on consumer electronics (GCCE). IEEE, pp 601–604 – reference: Nguyen-Duc A, Abrahamsson P (2020) Continuous experimentation on artificial intelligence software: a research agenda. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp 1513–1516 – reference: Hyun ParkSSeon ShinWHyun ParkYLeeYBuilding a new culture for quality management in the era of the fourth industrial revolutionTotal Qual Manag Bus Excell2017289-1093494510.1080/14783363.2017.1310703 – reference: Tsintzira AA, Arvanitou EM, Ampatzoglou A, Chatzigeorgiou A (2020) Applying machine learning in technical debt management: Future opportunities and challenges. In: International conference on the quality of information and communications technology. Springer, pp 53–67 – reference: Gezici B, Tarhan AK (2019) Final pool. https://drive.google.com/file/d/1ve6BpJTrITsfo6auSoWKh48ajWbNb05n/view?usp=sharing – reference: Tao C, Gao J, Wang T (2019) Testing and quality validation for ai software–perspectives, issues, and practices. IEEE Access 7:120164–120175 – reference: Aggarwal A, Lohia P, Nagar S, Dey K, Saha D (2019) Black box fairness testing of machine learning models. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp 625–635 – reference: Nakajima S (2019) Distortion and faults in machine learning software. In: International workshop on structured object-oriented formal language and method. Springer, pp 29–41 – reference: BraiekHBKhomhFOn testing machine learning programsJ Syst Softw202016411054210.1016/j.jss.2020.110542 – reference: Ishikawa F, Yoshioka N (2019) How do engineers perceive difficulties in engineering of machine-learning systems?-questionnaire survey. In: 2019 IEEE/ACM Joint 7th international workshop on conducting empirical studies in industry (CESI) and 6th international workshop on software engineering research and industrial practice (SER&IP). IEEE, pp 2–9 – reference: Lenarduzzi V, Lomio F, Moreschini S, Taibi D, Tamburri DA (2021) Software quality for ai: Where we are now?. In: International conference on software quality. Springer, pp 43–53 – reference: Byrne C (2017) Development Workflows for Data Scientists. O’Reilly Media – reference: 29119-1:2013 I (2013) Software and systems engineering — software testing. https://www.iso.org/standard/45142.html – reference: LwakatareLERajACrnkovicIBoschJOlssonHHLarge-scale machine learning systems in real-world industrial settings: a review of challenges and solutionsInf Softw Technol202012710636810.1016/j.infsof.2020.106368 – reference: Nakamichi K, Ohashi K, Namba I, Yamamoto R, Aoyama M, Joeckel L, Siebert J, Heidrich J (2020) Requirements-driven method to determine quality characteristics and measurements for machine learning software and its evaluation. In: 2020 IEEE 28Th international requirements engineering conference (RE). IEEE, pp 260–270 – reference: Pons L, Ozkaya I (2019) Priority quality attributes for engineering ai-enabled systems. arXiv:1911.02912 – reference: Russel S, Norvig P (2009) Artificial intelligence: a modern approach, English – reference: DengLArtificial intelligence in the rising wave of deep learning: The historical path and future outlook [perspectives]IEEE Signal Proc Mag201835118017710.1109/MSP.2017.2762725 – reference: Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering - – reference: Bosch J, Olsson HH, Crnkovic I (2021) Engineering ai systems: a research agenda. In: Artificial intelligence paradigms for smart cyber-physical systems. IGI Global, pp 1–19 – reference: MalikVSinghSArtificial intelligent environments: risk management and quality assurance implementationJ Discret Math Sci Cryptogr202023118719510.1080/09720529.2020.172188307477051 – reference: de Almeida BiolchiniJCMianPGNataliACCConteTTravassosGHScientific research ontology to support systematic review in software engineeringAdv Eng Inform200721213315110.1016/j.aei.2006.11.006 – reference: Wohlin C (2014) Guidelines for snowballing in systematic literature studies and a replication in software engineering. In: Proceedings of the 18th international conference on evaluation and assessment in software engineering, pp 1–10 – reference: 9126-1:2001 I (2001) Software engineering — product quality. https://www.iso.org/standard/22749.html – reference: Forward A, Lethbridge TC (2008) A taxonomy of software types to facilitate search and evidence-based software engineering. In: Proceedings of the 2008 conference of the center for advanced studies on collaborative research: meeting of minds, pp 179–191 – reference: Hamada K, Ishikawa F, Masuda S, Matsuya M, Ujita Y (2020) Guidelines for quality assurance of machine learning-based artificial intelligence. In: SEKE2020: The 32nd international conference on software engineering & knowledge engineering, pp 335–341 – reference: Poth A, Meyer B, Schlicht P, Riel A (2020) Quality assurance for machine learning–an approach to function and system safeguarding. In: 2020 IEEE 20Th international conference on software quality, reliability and security (QRS). IEEE, pp 22–29 – reference: Martínez-Fernández S, Bogner J, Franch X, Oriol M, Siebert J, Trendowicz A, Vollmer AM, Wagner S (2021) Software engineering for ai-based systems: A survey. arXiv:2105.01984 – reference: Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer Science & Business Media – reference: PetersenKVakkalankaSKuzniarzLGuidelines for conducting systematic mapping studies in software engineering: an updateInf Softw Technol20156411810.1016/j.infsof.2015.03.007 – reference: Masuda S, Ono K, Yasue T, Hosokawa N (2018) A survey of software quality for machine learning applications. In: 2018 IEEE International conference on software testing, verification and validation workshops (ICSTW). IEEE, pp 279–284 – reference: Taleb I, Serhani MA, Dssouli R (2018) Big data quality: a survey. In: 2018 IEEE International congress on big data (bigdata congress). IEEE, pp 166–173 – reference: Arpteg A, Brinne B, Crnkovic-Friis L, Bosch J (2018) Software engineering challenges of deep learning. In: 2018 44Th euromicro conference on software engineering and advanced applications (SEAA). IEEE, pp 50–59 – reference: ISO/IECIso/iec 25010 (2011)-systems and software quality requirements and evaluation (square)-system and software quality modelsInternational Standard ISO/IEC 25010201121125 – reference: Kuwajima H, Ishikawa F (2019) Adapting square for quality assessment of artificial intelligence systems. In: 2019 IEEE International symposium on software reliability engineering workshops (ISSREW). IEEE, pp 13–18 – reference: Ali Z Quality measurement challenges for artificial intelligence software – reference: KitchenhamBProcedures for performing systematic reviews. keele, UKKeele Univ2004332004126 – reference: Bourque P, Dupuis R, Abran A, Moore JW, Tripp L (2004) Guide to the software engineering body of knowledge - – reference: KuwajimaHYasuokaHNakaeTEngineering problems in machine learning systemsMach Learn2020109511031126410156810.1007/s10994-020-05872-w07224993 – reference: Alamin MAA, Uddin G (2021) Quality assurance challenges for machine learning software applications during software development life cycle phases. arXiv:2105.01195 – reference: Lakshen GA, Vraneš S., Janev V (2016) Big data and quality: A literature review. In: 2016 24Th telecommunications forum (TELFOR). IEEE, pp 1–4 – reference: Tao C, Hao C, Gao J, Wang T, Wen W (2017) A practical study on quality evaluation for age recognition systems. In: SEKE, pp 345–350 – reference: 26262-1:2018 I (2018) Road vehicles — functional safety. https://www.iso.org/standard/68383.html – reference: IvarssonMGorschekTA method for evaluating rigor and industrial relevance of technology evaluationsEmpir Softw Eng201116336539510.1007/s10664-010-9146-4 – reference: Henriksson J, Borg M, Englund C (2018) Automotive safety and machine learning: Initial results from a study on how to adapt the iso 26262 safety standard. In: 2018 IEEE/ACM 1St international workshop on software engineering for AI in autonomous systems (SEFAIAS). IEEE, pp 47–49 – reference: ChenRBastaniFBTsaoTWOn the reliability of ai planning software in real-time applicationsIEEE Trans Knowl Data Eng19957141310.1109/69.368522 – reference: Horkoff J (2019) Non-functional requirements for machine learning: Challenges and new directions. In: 2019 IEEE 27Th international requirements engineering conference (RE). IEEE, pp 386–391 – reference: Rushby J (1988) Quality measures and assurance for AI software, vol 18. National Aeronautics and Space Administration, Scientific and Technical Information Division – reference: Liu Y, Ma L, Zhao J (2019) Secure deep learning engineering: a road towards quality assurance of intelligent systems. In: International conference on formal engineering methods. Springer, pp 3–15 – reference: Zhang P, Cao W, Muccini H (2020) Quality assurance technologies of big data applications: A systematic literature review. arXiv:2002.01759 – reference: HannousseASearching relevant papers for software engineering secondary studies: Semantic scholar coverage and identification roleIET Softw202115112614610.1049/sfw2.12011 – reference: Samoili S, Cobo ML, Gomez E, De Prato G, Martinez-Plumed F, Delipetrev B (2020) Ai watch. defining artificial intelligence. towards an operational definition and taxonomy of artificial intelligence. In: JRC Technical reports. Joint research centre (seville site) – reference: Turhan B, Kutlubay O (2007) Mining software data. In: 2007 IEEE 23Rd international conference on data engineering workshop. IEEE, pp 912–916 – reference: 25012:2008 I (2008) software engineering — software product quality requirements and evaluation (square) — data quality model. https://www.iso.org/standard/35736.html – reference: Murphy C, Kaiser GE, Arias M (2006) A framework for quality assurance of machine learning applications - – reference: RiccioVJahangirovaGStoccoAHumbatovaNWeissMTonellaPTesting machine learning based systems: a systematic mappingEmpir Softw Eng20202565193525410.1007/s10664-020-09881-0 – reference: Wan Z, Xia X, Lo D, Murphy GC (2019) How does machine learning change software development practices? IEEE Transactions on Software Engineering – reference: Kuwajima H, Yasuoka H, Nakae T (2018) Open problems in engineering and quality assurance of safety critical machine learning systems. arXiv:1812.03057 – ident: 10105_CR52 doi: 10.1109/ICTKE.2017.8259629 – ident: 10105_CR17 doi: 10.1109/ICSME.2019.00051 – ident: 10105_CR44 doi: 10.1109/ICSTW.2018.00061 – volume: 65 start-page: 1 year: 2005 ident: 10105_CR26 publication-title: Adv Comput doi: 10.1016/S0065-2458(05)65001-2 – ident: 10105_CR12 doi: 10.4018/978-1-7998-5101-1.ch001 – ident: 10105_CR61 doi: 10.1007/978-3-030-58793-2_2 – volume: 33 start-page: 1 issue: 2004 year: 2004 ident: 10105_CR32 publication-title: Keele Univ – volume: 21 start-page: 133 issue: 2 year: 2007 ident: 10105_CR9 publication-title: Adv Eng Inform doi: 10.1016/j.aei.2006.11.006 – ident: 10105_CR1 – ident: 10105_CR51 doi: 10.1109/ICSTW.2018.00060 – ident: 10105_CR64 doi: 10.18293/SEKE2017-176 – ident: 10105_CR5 – ident: 10105_CR29 doi: 10.1109/CESSER-IP.2019.00009 – volume: 6 start-page: 835 issue: 6 year: 2007 ident: 10105_CR67 publication-title: Inf Technol J doi: 10.3923/itj.2007.835.842 – ident: 10105_CR37 doi: 10.1109/TELFOR.2016.7818902 – ident: 10105_CR34 doi: 10.1109/ISSREW.2019.00035 – ident: 10105_CR58 – ident: 10105_CR33 – volume: 127 start-page: 106368 year: 2020 ident: 10105_CR40 publication-title: Inf Softw Technol doi: 10.1016/j.infsof.2020.106368 – ident: 10105_CR54 – volume: 28 start-page: 934 issue: 9-10 year: 2017 ident: 10105_CR28 publication-title: Total Qual Manag Bus Excell doi: 10.1080/14783363.2017.1310703 – ident: 10105_CR4 – ident: 10105_CR13 – volume: 35 start-page: 180 issue: 1 year: 2018 ident: 10105_CR18 publication-title: IEEE Signal Proc Mag doi: 10.1109/MSP.2017.2762725 – ident: 10105_CR63 doi: 10.1109/ACCESS.2019.2937107 – ident: 10105_CR6 doi: 10.1145/3338906.3338937 – volume: 25 start-page: 5193 issue: 6 year: 2020 ident: 10105_CR57 publication-title: Empir Softw Eng doi: 10.1007/s10664-020-09881-0 – ident: 10105_CR42 doi: 10.1145/3371158.3371233 – ident: 10105_CR23 – ident: 10105_CR8 – ident: 10105_CR46 doi: 10.1109/GCCE.2018.8574766 – volume: 7 start-page: 4 issue: 1 year: 1995 ident: 10105_CR16 publication-title: IEEE Trans Knowl Data Eng doi: 10.1109/69.368522 – ident: 10105_CR48 doi: 10.1109/RE48521.2020.00036 – ident: 10105_CR59 – ident: 10105_CR74 – ident: 10105_CR65 doi: 10.1007/978-3-030-58793-2_5 – ident: 10105_CR68 doi: 10.1109/REW.2019.00050 – ident: 10105_CR50 doi: 10.1145/3368089.3417039 – volume: 164 start-page: 110542 year: 2020 ident: 10105_CR14 publication-title: J Syst Softw doi: 10.1016/j.jss.2020.110542 – ident: 10105_CR71 doi: 10.1145/2601248.2601268 – ident: 10105_CR3 – ident: 10105_CR10 doi: 10.1109/SEAA.2018.00018 – ident: 10105_CR66 doi: 10.1109/ICDEW.2007.4401084 – ident: 10105_CR38 doi: 10.1007/978-3-030-65854-0_4 – volume: 15 start-page: 126 issue: 1 year: 2021 ident: 10105_CR24 publication-title: IET Softw doi: 10.1049/sfw2.12011 – ident: 10105_CR11 doi: 10.2991/jase.d.190131.001 – ident: 10105_CR56 doi: 10.1109/EIT48999.2020.9208288 – volume: 23 start-page: 187 issue: 1 year: 2020 ident: 10105_CR41 publication-title: J Discret Math Sci Cryptogr doi: 10.1080/09720529.2020.1721883 – ident: 10105_CR60 – ident: 10105_CR72 doi: 10.1007/978-3-642-29044-2 – ident: 10105_CR22 – ident: 10105_CR49 – volume: 109 start-page: 1103 issue: 5 year: 2020 ident: 10105_CR36 publication-title: Mach Learn doi: 10.1007/s10994-020-05872-w – ident: 10105_CR45 – ident: 10105_CR7 – ident: 10105_CR19 doi: 10.1145/1463788.1463807 – volume: 2 start-page: 1 issue: 1 year: 2011 ident: 10105_CR30 publication-title: International Standard ISO/IEC 25010 – ident: 10105_CR35 – ident: 10105_CR43 doi: 10.1145/3487043 – ident: 10105_CR20 doi: 10.1145/2915970.2916008 – ident: 10105_CR70 doi: 10.1007/978-3-662-43839-8 – volume: 64 start-page: 1 year: 2015 ident: 10105_CR53 publication-title: Inf Softw Technol doi: 10.1016/j.infsof.2015.03.007 – ident: 10105_CR73 – ident: 10105_CR39 doi: 10.1007/978-3-030-32409-4_1 – ident: 10105_CR47 doi: 10.1007/978-3-030-41418-4_3 – ident: 10105_CR2 – ident: 10105_CR15 – ident: 10105_CR55 doi: 10.1109/QRS51102.2020.00016 – ident: 10105_CR62 doi: 10.1109/BigDataCongress.2018.00029 – ident: 10105_CR21 – ident: 10105_CR69 doi: 10.1109/TSE.2019.2937083 – ident: 10105_CR25 doi: 10.1145/3194085.3194090 – ident: 10105_CR27 doi: 10.1109/RE.2019.00050 – volume: 16 start-page: 365 issue: 3 year: 2011 ident: 10105_CR31 publication-title: Empir Softw Eng doi: 10.1007/s10664-010-9146-4 |
| SSID | ssj0009745 |
| Score | 2.400237 |
| Snippet | There is a widespread demand for Artificial Intelligence (AI) software, specifically Machine Learning (ML). It is getting increasingly popular and being... |
| SourceID | proquest crossref springer |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| SubjectTerms | Artificial intelligence Compilers Computer Science Empirical analysis Interpreters Literature reviews Machine learning Programming Languages Quality management Software Software Engineering/Programming and Operating Systems Software quality Systematic review |
| SummonAdditionalLinks | – databaseName: ProQuest Central (New) dbid: BENPR link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV07T8MwELagXVh4IwoFeWADi9pJHHdCBbUqSFQIqNQt8nNCaaFFiH_POXESgUS3KE483Nl339l33yF04bhONawEkgIeILHqx0RKbgg1IqbWSAANvlD4ccLH0_hhlszCgdsypFVWNrEw1Gau_Rn5NThu8NVgbpObxTvxXaP87WpoobGJ2mCCBQRf7dvh5Om5od1NizbFnmiPRODdQ9lMKJ7jPCY-RYH6NpGE_XZNDd78c0VaeJ7RLtoOkBEPSh3voQ2b76Odqh0DDrvzAI1ealZm_FazJeOyOAXPc7wEk_sl4VVZSfmNAbDiwT3xnszUo4doOhq-3o1JaJNAdCSSFeFK9XuGuThJUi1d7ACT8MhKC7GdUz1tDaWeZqgvBAQbihkICakWTEoAX87p6Ai18nlujxG2oDYnhZAMoiijKDy41DipjNFMC9dBtJJQpgOHuG9l8ZY17MdeqhlINSukmrEOuqz_WZQMGmu_7laCz8JuWmaN7jvoqlJGM_z_bCfrZztFW8xXMxT5i13UWn182jPAGCt1HhbSD7LEzPA priority: 102 providerName: ProQuest |
| Title | Systematic literature review on software quality for AI-based software |
| URI | https://link.springer.com/article/10.1007/s10664-021-10105-2 https://www.proquest.com/docview/2640563625 |
| Volume | 27 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA7aXrz4Fqu15OBNA032lR5X6bYqFlEL9bTkeSpbsRXx3zvZp4oKnnbZZHOYPOYbMt83CJ3aUEUKVgKJAA8QXw58IkSoCdXcp0YLAA2OKHw7CcdT_3oWzEpS2LLKdq-uJPOT-hPZLQx94lIKqCvrSODgbQcudodVPGVxI7Ub5aWJnbge8cCjl1SZn8f46o4ajPntWjT3Nsk22ixhIo6Led1BaybbRVtVCQZc7sg9lDzUSsx4Xisk44KQghcZXsIx-ybgU8GefMcAUnF8RZz30nXrPpomw8fLMSlLIxDl8WBFQikHfc2sHwSREta3gENCzwgD8ZyVfWU0pU5aaMA5BBiSaQgDqeJMCABc1irvALWyRWYOETYwVVZwLhhETlpSeLGRtkJqrZjitoNoZaFUlbrhrnzFPG0Uj51VU7Bqmls1ZR10Vv_zXKhm_Nm7Wxk-LXfQMgWgBtgM3GvQQefVZDTNv4929L_ux2iDOUZDnsPYRa3Vy6s5AZyxkj20zpNRD7Xj0dPNEJ4Xw8ndfS9fbB-Ubsxv |
| linkProvider | Springer Nature |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV25TgMxEB1xFNBwIwIBXEAFFlnv5RQIIWBJyNEQJLrF66NCCZAglJ_iGxnvKZBIl261h4vx88ybtecNwIkJZCgRCTREPkC9pOlRIQJFHcU9RyuBpMEWCvf6QevJe3j2nxfgu6iFsccqC5-YOmo1kvYf-QUGbozV6G79q7d3artG2d3VooVGBouOnn5hyja-bN_i_J4yFt0Nblo07ypApcv9CQ2SpNlQzHi-H0phPIMhPHC10JgKmaQhtXIcq8rT5By5ecIUZlCO5EwI5CrGSBfHXYRlz3WbdkXx6L4S-Q3TpshW1o-6yCXyIp28VC8IPGoPRDi2KSVlvwNhxW7_bMimcS7agLWcoJLrDFGbsKCHW7BeNH8guS_Yhuix1IAmr6U2M8lKYchoSMbo4L8E3srqNqcE6TG5blMbN1X5dAee5mK-XVgajoZ6D4hGkBjBuWCYs6nEwQsTKiMSpSST3NTAKSwUy1yx3DbOeI0rrWVr1RitGqdWjVkNzspv3jK9jplv1wvDx_naHccV0mpwXkxG9fj_0fZnj3YMK61Brxt32_3OAawyW0eRnpysw9Lk41MfIruZJEcppAi8zBvDP-77CDk |
| linkToPdf | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1LT8JAEJ4gJsaLbyOKugc96Ua6fXIwBhUCosSoJN7qdh8nAyoYwl_z1znbbmk0kZu3ptvuYfbrzDfdnW8AjnQgQoFIoCHyAeoldY9yHkjqyMhzlORIGkyh8F0vaPe9m2f_uQRfeS2MOVaZ-8TUUcuhMP_IzzBwY6xGd-ufaXss4v66dfH2Tk0HKbPTmrfTyCDSVdMJpm-j8841rvUxY63m01Wb2g4DVLiRP6ZBktRrkmnP90PBtacxnAeu4grTIp3UhJKOYxR66lGEPD1hErMpR0SMc-QtWgsX512AxRCzoloZFi-bvfuHQvI3TFskG5E_6iKzsCU7tnAvCDxqjkc4pkUlZT_DYsF1f23PplGvtQYrlq6SRoavdSipwQas5q0giPUMm9B6nClCk9eZUjPJCmPIcEBG6O4nHG9lVZxTgmSZNDrURFE5G92C_r8YcBvKg-FA7QBRCBnNo4gzzOBk4uCFDqXmiZSCiUhXwMktFAurX27aaLzGhfKysWqMVo1Tq8asAiezd94y9Y65T1dzw8f2Sx7FBe4qcJovRjH892y782c7hCXEb3zb6XX3YJmZoor0GGUVyuOPT7WPVGecHFhMEXj5bxh_A8HPDcs |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Systematic+literature+review+on+software+quality+for+AI-based+software&rft.jtitle=Empirical+software+engineering+%3A+an+international+journal&rft.au=Gezici%2C+Bahar&rft.au=Tarhan%2C+Ay%C3%A7a+Koluk%C4%B1sa&rft.date=2022-05-01&rft.pub=Springer+US&rft.issn=1382-3256&rft.eissn=1573-7616&rft.volume=27&rft.issue=3&rft_id=info:doi/10.1007%2Fs10664-021-10105-2&rft.externalDocID=10_1007_s10664_021_10105_2 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1382-3256&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1382-3256&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1382-3256&client=summon |