deepmind
Artificial superintelligence and its limits: why AlphaZero cannot become a general agent
AI & Society (2020) Abstract An intelligent machine surpassing human intelligence across a wide set of skills has been proposed as a possible existential catastrophe (i.e., an event comparable in val
Artificial superintelligence and its limits: why AlphaZero cannot become a general agent
in: AI & SOCIETY AbstractAn intelligent machine surpassing human intelligence across a wide set of skills has been proposed as a possible existential catastrophe (i.e., an event comparable in valueproductivedesires, or desires that can direct behavior across multiple contexts. However, productive desires cannotsui generisbe derived from non-productive desires. Thus, even though general agency in AI could in principle be created by human agents, general agency cannot be spontaneously produced by a non-general AI agent through an endogenous process (i.e. self-improvement). In conclusion, we argue that a common AI scenario, where general agency suddenly emerges in a non-general agent AI, such as DeepMind’s superintelligent board game AI AlphaZero, is not plausible.
