Seq-DeepIPC: Sequential Sensing for End-to-End Control in Legged Robot Navigation

TL;DR

Seq-DeepIPC is a novel end-to-end perception-to-control model for legged robot navigation that integrates multi-modal sensing, temporal fusion, and efficient processing, enabling robust navigation in diverse real-world environments.

Contribution

The paper introduces Seq-DeepIPC, a new sequential perception-to-control model that combines multi-modal perception with temporal fusion for improved legged robot navigation.

Findings

Sequential inputs enhance perception and control accuracy.

EfficientNet-B0 enables real-time deployment on edge devices.

GNSS-based heading estimation is effective in open areas.

Abstract

We present Seq-DeepIPC, a sequential end-to-end perception-to-control model for legged robot navigation in realworld environments. Seq-DeepIPC advances intelligent sensing for autonomous legged navigation by tightly integrating multi-modal perception (RGB-D + GNSS) with temporal fusion and control. The model jointly predicts semantic segmentation and depth estimation, giving richer spatial features for planning and control. For efficient deployment on edge devices, we use EfficientNet-B0 as the encoder, reducing computation while maintaining accuracy. Heading estimation is simplified by removing the noisy IMU and instead computing the bearing angle directly from consecutive GNSS positions. We collected a larger and more diverse dataset that includes both road and grass terrains, and validated Seq-DeepIPC on a robot dog. Comparative and ablation studies show that sequential inputs improve perception and control in our models, while other baselines do not benefit. Seq-DeepIPC achieves competitive or better results with reasonable model size; although GNSS-only heading is less reliable near tall buildings, it is robust in open areas. Overall, Seq-DeepIPC extends end-to-end navigation beyond wheeled robots to more versatile and temporally-aware systems. To support future research, we will release the codes to our GitHub repository at https://github.com/oskarnatan/Seq-DeepIPC.