Using benchmarking to advance research: a challenge to software engineering
Benchmarks have been used in computer science to compare the performance of computer systems, information retrieval algorithms, databases, and many other technologies. The creation and widespread use of a benchmark within a research area is frequently accompanied by rapid technical progress and comm...
Saved in:
Published in | 25th International Conference on Software Engineering, 2003. Proceedings pp. 74 - 83 |
---|---|
Main Authors | , , |
Format | Conference Proceeding |
Language | English |
Published |
IEEE
2003
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Benchmarks have been used in computer science to compare the performance of computer systems, information retrieval algorithms, databases, and many other technologies. The creation and widespread use of a benchmark within a research area is frequently accompanied by rapid technical progress and community building. These observations have led us to formulate a theory of benchmarking within scientific disciplines. Based on this theory, we challenge software engineering research to become more scientific and cohesive by working as a community to define benchmarks. In support of this challenge, we present a case study of the reverse engineering community, where we have successfully used benchmarks to advance the state of research. |
---|---|
ISBN: | 076951877X 9780769518770 |
ISSN: | 0270-5257 1558-1225 |
DOI: | 10.1109/ICSE.2003.1201189 |