A Systematic Review of Theory Use in Software Engineering Experiments

Empirically based theories are generally perceived as foundational to science. However, in many disciplines, the nature, role and even the necessity of theories remain matters for debate, particularly in young or practical disciplines such as software engineering. This article reports a systematic r...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on software engineering Vol. 33; no. 2; pp. 87 - 107
Main Authors Hannay, J.E., Sjoberg, D.I.K., Dyba, T.
Format Journal Article
LanguageEnglish
Published New York IEEE 01.02.2007
IEEE Computer Society
Subjects
Computer industry
Conferences
Design engineering
Design for experiments
Empirical analysis
empirical software engineering
Experiments
Hypotheses
Knowledge
Laboratories
Literature
Programming
Reporting
research methodology
Software engineering
Studies
Systematic review
Testing
Theory
Online AccessGet full text

Cover

More Information
Summary:Empirically based theories are generally perceived as foundational to science. However, in many disciplines, the nature, role and even the necessity of theories remain matters for debate, particularly in young or practical disciplines such as software engineering. This article reports a systematic review of the explicit use of theory in a comprehensive set of 103 articles reporting experiments, from of a total of 5,453 articles published in major software engineering journals and conferences in the decade 1993-2002. Of the 103 articles, 24 use a total of 40 theories in various ways to explain the cause-effect relationship(s) under investigation. The majority of these use theory in the experimental design to justify research questions and hypotheses, some use theory to provide post hoc explanations of their results, and a few test or modify theory. A third of the theories are proposed by authors of the reviewed articles. The interdisciplinary nature of the theories used is greater than that of research in software engineering in general. We found that theory use and awareness of theoretical issues are present, but that theory-driven research is, as yet, not a major issue in empirical software engineering. Several articles comment explicitly on the lack of relevant theory. We call for an increased awareness of the potential benefits of involving theory, when feasible. To support software engineering researchers who wish to use theory, we show which of the reviewed articles on which topics use which theories for what purposes, as well as details of the theories' characteristics
Bibliography:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 23
ISSN:0098-5589
1939-3520
DOI:10.1109/TSE.2007.12