Embodied Intelligence via Learning and Evolution

TL;DR

This paper introduces DERL, a framework combining evolution and learning to develop diverse agent morphologies that improve task learnability in complex environments, revealing links between environmental complexity, morphology, and intelligence.

Contribution

The paper presents DERL, a novel framework that evolves agent morphologies to enhance learning in complex settings, demonstrating the morphological Baldwin effect and insights into physical stability and energy efficiency.

Findings

Environmental complexity promotes morphological intelligence.

Evolution favors morphologies that learn faster.

Emergence of the Baldwin effect in evolving agents.

Abstract

The intertwined processes of learning and evolution in complex environmental niches have resulted in a remarkable diversity of morphological forms. Moreover, many aspects of animal intelligence are deeply embodied in these evolved morphologies. However, the principles governing relations between environmental complexity, evolved morphology, and the learnability of intelligent control, remain elusive, partially due to the substantial challenge of performing large-scale in silico experiments on evolution and learning. We introduce Deep Evolutionary Reinforcement Learning (DERL): a novel computational framework which can evolve diverse agent morphologies to learn challenging locomotion and manipulation tasks in complex environments using only low level egocentric sensory information. Leveraging DERL we demonstrate several relations between environmental complexity, morphological intelligence and the learnability of control. First, environmental complexity fosters the evolution of morphological intelligence as quantified by the ability of a morphology to facilitate the learning of novel tasks. Second, evolution rapidly selects morphologies that learn faster, thereby enabling behaviors learned late in the lifetime of early ancestors to be expressed early in the lifetime of their descendants. In agents that learn and evolve in complex environments, this result constitutes the first demonstration of a long-conjectured morphological Baldwin effect. Third, our experiments suggest a mechanistic basis for both the Baldwin effect and the emergence of morphological intelligence through the evolution of morphologies that are more physically stable and energy efficient, and can therefore facilitate learning and control.