Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)
The recent paper `"Reward is Enough" by Silver, Singh, Precup and Sutton posits that the concept of reward maximisation is sufficient to underpin all intelligence, both natural and artificial. We contest the underlying assumption of Silver et al. that such reward can be scalar-valued. In t...
Saved in:
Main Authors | , , , , , , , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
24.11.2021
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | The recent paper `"Reward is Enough" by Silver, Singh, Precup and Sutton
posits that the concept of reward maximisation is sufficient to underpin all
intelligence, both natural and artificial. We contest the underlying assumption
of Silver et al. that such reward can be scalar-valued. In this paper we
explain why scalar rewards are insufficient to account for some aspects of both
biological and computational intelligence, and argue in favour of explicitly
multi-objective models of reward maximisation. Furthermore, we contend that
even if scalar reward functions can trigger intelligent behaviour in specific
cases, it is still undesirable to use this approach for the development of
artificial general intelligence due to unacceptable risks of unsafe or
unethical behaviour. |
---|---|
DOI: | 10.48550/arxiv.2112.15422 |