Using benchmarking to advance research: a challenge to software engineering

Benchmarks have been used in computer science to compare the performance of computer systems, information retrieval algorithms, databases, and many other technologies. The creation and widespread use of a benchmark within a research area is frequently accompanied by rapid technical progress and comm...

Full description

Saved in:
Bibliographic Details
Published in25th International Conference on Software Engineering, 2003. Proceedings pp. 74 - 83
Main Authors Sim, S.E., Easterbrook, S., Holt, R.C.
Format Conference Proceeding
LanguageEnglish
Published IEEE 2003
Subjects
Collaboration
Computer languages
Computer science
Guidelines
Information retrieval
Reverse engineering
Software algorithms
Software engineering
Online AccessGet full text

Cover

More Information
Summary:Benchmarks have been used in computer science to compare the performance of computer systems, information retrieval algorithms, databases, and many other technologies. The creation and widespread use of a benchmark within a research area is frequently accompanied by rapid technical progress and community building. These observations have led us to formulate a theory of benchmarking within scientific disciplines. Based on this theory, we challenge software engineering research to become more scientific and cohesive by working as a community to define benchmarks. In support of this challenge, we present a case study of the reverse engineering community, where we have successfully used benchmarks to advance the state of research.
ISBN:076951877X
9780769518770
ISSN:0270-5257
1558-1225
DOI:10.1109/ICSE.2003.1201189