Persistent-Transient Duality: A Multi-mechanism Approach for Modeling Human-Object Interaction

TL;DR

This paper introduces a dual-mechanism neural network model that captures both continuous global plans and intermittent local actions in human-object interactions, improving motion forecasting accuracy.

Contribution

It proposes a novel Persistent-Transient Duality framework with dedicated neural modules for dynamic switching, addressing limitations of previous morphing structure models.

Findings

Outperforms existing models on two rich datasets

Effectively captures mode-switching in human motion

Demonstrates superior motion forecasting accuracy

Abstract

Humans are highly adaptable, swiftly switching between different modes to progressively handle different tasks, situations and contexts. In Human-object interaction (HOI) activities, these modes can be attributed to two mechanisms: (1) the large-scale consistent plan for the whole activity and (2) the small-scale children interactive actions that start and end along the timeline. While neuroscience and cognitive science have confirmed this multi-mechanism nature of human behavior, machine modeling approaches for human motion are trailing behind. While attempted to use gradually morphing structures (e.g., graph attention networks) to model the dynamic HOI patterns, they miss the expeditious and discrete mode-switching nature of the human motion. To bridge that gap, this work proposes to model two concurrent mechanisms that jointly control human motion: the Persistent process that runs continually on the global scale, and the Transient sub-processes that operate intermittently on the local context of the human while interacting with objects. These two mechanisms form an interactive Persistent-Transient Duality that synergistically governs the activity sequences. We model this conceptual duality by a parent-child neural network of Persistent and Transient channels with a dedicated neural module for dynamic mechanism switching. The framework is trialed on HOI motion forecasting. On two rich datasets and a wide variety of settings, the model consistently delivers superior performances, proving its suitability for the challenge.