HVIS: A Human-like Vision and Inference System for Human Motion Prediction

TL;DR

HVIS is a novel system that emulates human visual and inference processes to improve the accuracy of human motion prediction, achieving state-of-the-art results on multiple datasets.

Contribution

The paper introduces HVIS, combining a human-like vision encoding and a multi-stage motion inference model inspired by human cognition, advancing human motion prediction techniques.

Findings

Achieves 19.8% improvement on Human3.6M dataset.

Outperforms existing methods by 15.7% on CMU Mocap.

Surpasses previous approaches by 11.1% on G3D dataset.

Abstract

Grasping the intricacies of human motion, which involve perceiving spatio-temporal dependence and multi-scale effects, is essential for predicting human motion. While humans inherently possess the requisite skills to navigate this issue, it proves to be markedly more challenging for machines to emulate. To bridge the gap, we propose the Human-like Vision and Inference System (HVIS) for human motion prediction, which is designed to emulate human observation and forecast future movements. HVIS comprises two components: the human-like vision encode (HVE) module and the human-like motion inference (HMI) module. The HVE module mimics and refines the human visual process, incorporating a retina-analog component that captures spatiotemporal information separately to avoid unnecessary crosstalk. Additionally, a visual cortex-analogy component is designed to hierarchically extract and treat complex motion features, focusing on both global and local features of human poses. The HMI is employed to simulate the multi-stage learning model of the human brain. The spontaneous learning network simulates the neuronal fracture generation process for the adversarial generation of future motions. Subsequently, the deliberate learning network is optimized for hard-to-train joints to prevent misleading learning. Experimental results demonstrate that our method achieves new state-of-the-art performance, significantly outperforming existing methods by 19.8% on Human3.6M, 15.7% on CMU Mocap, and 11.1% on G3D.