The Effect of Training Schedules on Morphological Robustness and Generalization

TL;DR

This paper investigates how different training schedules affect the morphological robustness and generalization of ANN controllers, using evolutionary learning and reinforcement learning to optimize variability introduction.

Contribution

It introduces various training schedules for morphological variations and formalizes sample selection as a reinforcement learning problem to enhance generalization.

Findings

Training schedules significantly impact robustness and generalization.

Reinforcement learning can optimize morphological variation sampling.

Certain schedules lead to more adaptable ANN controllers.

Abstract

Robustness and generalizability are the key properties of artificial neural network (ANN)-based controllers for maintaining a reliable performance in case of changes. It is demonstrated that exposing the ANNs to variations during training processes can improve their robustness and generalization capabilities. However, the way in which this variation is introduced can have a significant impact. In this paper, we define various training schedules to specify how these variations are introduced during an evolutionary learning process. In particular, we focus on morphological robustness and generalizability concerned with finding an ANN-based controller that can provide sufficient performance on a range of physical variations. Then, we perform an extensive analysis of the effect of these training schedules on morphological generalization. Furthermore, we formalize the process of training sample selection (i.e., morphological variations) to improve generalization as a reinforcement learning problem. Overall, our results provide deeper insights into the role of variability and the ways of enhancing the generalization property of evolved ANN-based controllers.