An Intelligent Social Learning-based Optimization Strategy for Black-box Robotic Control with Reinforcement Learning

TL;DR

This paper introduces an Intelligent Social Learning algorithm that enhances black-box robotic control by mimicking social learning behaviors, integrating reinforcement learning principles, and demonstrating superior performance in benchmarks and real robot tasks.

Contribution

The paper presents a novel ISL algorithm combining social learning styles with reinforcement learning for black-box robot control, showing improved efficiency and effectiveness.

Findings

ISL outperforms four state-of-the-art methods on six benchmarks.

ISL achieves satisfactory results in UR3 robot grasping tasks.

The algorithm exhibits fast computation and robustness to sparse rewards.

Abstract

Implementing intelligent control of robots is a difficult task, especially when dealing with complex black-box systems, because of the lack of visibility and understanding of how these robots work internally. This paper proposes an Intelligent Social Learning (ISL) algorithm to enable intelligent control of black-box robotic systems. Inspired by mutual learning among individuals in human social groups, ISL includes learning, imitation, and self-study styles. Individuals in the learning style use the Levy flight search strategy to learn from the best performer and form the closest relationships. In the imitation style, individuals mimic the best performer with a second-level rapport by employing a random perturbation strategy. In the self-study style, individuals learn independently using a normal distribution sampling method while maintaining a distant relationship with the best performer. Individuals in the population are regarded as autonomous intelligent agents in each style. Neural networks perform strategic actions in three styles to interact with the environment and the robot and iteratively optimize the network policy. Overall, ISL builds on the principles of intelligent optimization, incorporating ideas from reinforcement learning, and possesses strong search capabilities, fast computation speed, fewer hyperparameters, and insensitivity to sparse rewards. The proposed ISL algorithm is compared with four state-of-the-art methods on six continuous control benchmark cases in MuJoCo to verify its effectiveness and advantages. Furthermore, ISL is adopted in the simulation and experimental grasping tasks of the UR3 robot for validations, and satisfactory solutions are yielded.