Multi-objective Performance Measurement: Alternatives to PAR10 and Expected Running Time - Bossek, Jakob; Trautmann, Heike (Learning and Intelligent Optimization, 2019)
A multiobjective perspective onto common performance measures such as the PAR10 score or the expected runtime of single-objective stochastic solvers is presented by directly investigating the tradeoff between the fraction of failed runs and the average runtime. Multi-objective indicators operating in the bi-objective space allow for an overall performance comparison on a set of instances paving the way for instance-based automated algorithm selection techniques.