Learning a Vision-Based Footstep Planner for Hierarchical Walking Control

TL;DR

This paper introduces a vision-based hierarchical control system for bipedal robots that combines reinforcement learning for footstep planning with a low-level controller, improving navigation in unstructured terrains.

Contribution

It presents a novel integrated framework using reinforcement learning and a low-dimensional dynamic model for real-time footstep planning in complex environments.

Findings

Effective navigation on challenging terrains demonstrated in simulation.

Successful hardware implementation on Cassie robot.

Robustness of the approach in unstructured environments.

Abstract

Bipedal robots demonstrate potential in navigating challenging terrains through dynamic ground contact. However, current frameworks often depend solely on proprioception or use manually designed visual pipelines, which are fragile in real-world settings and complicate real-time footstep planning in unstructured environments. To address this problem, we present a vision-based hierarchical control framework that integrates a reinforcement learning high-level footstep planner, which generates footstep commands based on a local elevation map, with a low-level Operational Space Controller that tracks the generated trajectories. We utilize the Angular Momentum Linear Inverted Pendulum model to construct a low-dimensional state representation to capture an informative encoding of the dynamics while reducing complexity. We evaluate our method across different terrain conditions using the underactuated bipedal robot Cassie and investigate the capabilities and challenges of our approach through simulation and hardware experiments.