Decentralized Circle Formation Control for Fish-like Robots in the Real-world via Reinforcement Learning

TL;DR

This paper presents a decentralized reinforcement learning-based control method enabling fish-like robots to form circles in real-world environments without prior knowledge of their dynamics, demonstrating scalability and robustness.

Contribution

It introduces a novel model-free reinforcement learning controller for decentralized circle formation in fish-like robots, capable of transferring from simulation to real-world without manual tuning.

Findings

Controller successfully deployed on real robots

Outperforms other RL algorithms in simulations

Scalable with group size of robots

Abstract

In this paper, the circle formation control problem is addressed for a group of cooperative underactuated fish-like robots involving unknown nonlinear dynamics and disturbances. Based on the reinforcement learning and cognitive consistency theory, we propose a decentralized controller without the knowledge of the dynamics of the fish-like robots. The proposed controller can be transferred from simulation to reality. It is only trained in our established simulation environment, and the trained controller can be deployed to real robots without any manual tuning. Simulation results confirm that the proposed model-free robust formation control method is scalable with respect to the group size of the robots and outperforms other representative RL algorithms. Several experiments in the real world verify the effectiveness of our RL-based approach for circle formation control.