Robot motor learning shows emergence of frequency-modulated, robust swimming with an invariant Strouhal-number

TL;DR

This study demonstrates that robot models of fish swimming develop robust, frequency-modulated gaits with an invariant Strouhal number through reinforcement learning, revealing fundamental principles of fluid-structure interaction in locomotion.

Contribution

The paper introduces a robotic framework that uses reinforcement learning to uncover how motor frequency and fluid-structure interactions lead to robust, invariant swimming gaits with a constant Strouhal number.

Findings

Motor frequency converges faster than other parameters during learning.

Emergent swimming gaits are robust against motor control disruptions.

Swim speed correlates with undulation velocity, maintaining an invariant Strouhal number.

Abstract

Fish locomotion emerges from a diversity of interactions among deformable structures, surrounding fluids and neuromuscular activations, i.e., fluid-structure interactions (FSI) controlled by fish's motor systems. Previous studies suggested that such motor-controlled FSI may possess embodied traits. However, their implications in motor learning, neuromuscular control, gait generation, and swimming performance remain to be uncovered. Using robot models, we studied how swimming behaviours emerged from the FSI and the embodied traits. We developed modular robots with various designs and used Central Pattern Generators (CPGs) to control the torque acting on robot body. We used reinforcement learning to learn CPG parameters to maximize the swimming speed. The results showed that motor frequency converged faster than other parameters, and the emergent swimming gaits were robust against disruptions applied to motor control. For all robots and frequencies tested, swimming speed was proportional to the mean undulation velocity of body and caudal-fin combined, yielding an invariant, undulation-based Strouhal number. The Strouhal number also revealed two fundamental classes of undulatory swimming in both biological and robotic fishes. The robot actuators also demonstrated diverse functions as motors, virtual springs, and virtual masses. These results provide novel insights into the embodied traits of motor-controlled FSI for fish-inspired locomotion.