Incremental Neuroevolution of Reactive and Deliberative 3D Agents

Abstract

Following earlier work on the neuroevolution of deliberative behaviour to solve increasingly challenging tasks in a twodimensional dynamic world, this paper presents the results of extending the original system to a three-dimensional rigid body simulation. The 3D physically based setting requires that a successful agent continually and deliberately adjust its gait, turning and other motor control over the many stages and sub-stages of these tasks, within its individual evaluation. Achieving such complex interplay between motor control and deliberative control, within a neuroevolutionary framework, is the focus of this work. To this end, a novel neural architecture is presented and an incremental evolutionary approach used to bootstrap the locomotive behaviour of the agents. Agent morphology is fixed as a quadruped with three degrees of freedom per limb. Agent populations have no initial knowledge of the problem domain, and evolve to move around and then solve progressively more difficult challenges in the environment using a tournament-based co-evolutionary algorithm. The results demonstrate not only success at the tasks but also a variety of intricate lifelike behaviours being used, separately and in combination, to achieve this success. Given the problem-agnostic controller architecture, these results indicate a potential for discovering yet more advanced behaviours in yet more complex environments.