Using Common Random Numbers for Indifference-Zone Selection and Multiple Comparisons in Simulation

• Published in: Management science Vol. 41; no. 12; pp. 1935 - 1945
• Main Authors: Nelson, Barry L, Matejcik, Frank J
• Format: Journal Article
• Language: English
• Published: Linthicum, MD INFORMS 01.12.1995; Institute for Operations Research and the Management Sciences
• Series: Management Science
• Subjects: Applications; Biology, psychology, social sciences; common random numbers; Comparative analysis; Confidence interval; Correlations; Estimators; Exact sciences and technology; Inference; Input-output analysis; Insurance, economics, finance; Management science; Mathematical procedures; Mathematics; Medical sciences; multiple comparisons; output analysis; Period costs; Probability and statistics; Random number sampling; Random numbers; Reliability, life testing, quality control; Sample mean; Sample size; Sciences and techniques of general use; Simulation; Statistical variance; Statistics; Stochastic models; Studies; Variance analysis; variance reduction; Selection problem; Sphericity; Sample size; Inventory control; Stochastic simulation; System simulation; Multiple comparison; Output analysis; Indifference zone
• ISSN: 0025-1909; 1526-5501
• DOI: 10.1287/mnsc.41.12.1935

We present a general recipe for constructing experiment design and analysis procedures that simultaneously provide indifference-zone selection and multiple-comparison inference for choosing the best among k simulated systems. We then exhibit two such procedures that exploit the variance-reduction technique of common random numbers to reduce the sample size required to attain a fixed precision. One procedure is based on the Bonferroni inequality and is guaranteed to be statistically conservative. The other procedure is exact under a specific dependence structure, but may be slightly liberal otherwise. Both are easy to apply, requiring only simple calculations and tabled constants. We illustrate the procedures with a numerical example.