Towards Autonomous Pipeline Inspection with Hierarchical Reinforcement Learning

TL;DR

This paper explores using Hierarchical Reinforcement Learning to enable autonomous navigation of in-pipe robots in complex pipeline networks, achieving performance surpassing human control.

Contribution

It introduces a hierarchical policy framework for in-pipe robot navigation, demonstrating its effectiveness over flat RL approaches.

Findings

Hierarchical RL significantly improves navigation success rates.

The approach outperforms human-level control in complex pipeline scenarios.

Hierarchical structure is essential for robust in-pipe navigation.

Abstract

Inspection and maintenance are two crucial aspects of industrial pipeline plants. While robotics has made tremendous progress in the mechanic design of in-pipe inspection robots, the autonomous control of such robots is still a big open challenge due to the high number of actuators and the complex manoeuvres required. To address this problem, we investigate the usage of Deep Reinforcement Learning for achieving autonomous navigation of in-pipe robots in pipeline networks with complex topologies. Moreover, we introduce a hierarchical policy decomposition based on Hierarchical Reinforcement Learning to learn robust high-level navigation skills. We show that the hierarchical structure introduced in the policy is fundamental for solving the navigation task through pipes and necessary for achieving navigation performances superior to human-level control.