IntentVLM: Open-Vocabulary Intention Recognition through Forward-Inverse Modeling with Video-Language Models

TL;DR

IntentVLM introduces a two-stage video-language framework for open-vocabulary intention recognition, significantly improving accuracy in human-robot interaction by reducing hallucinations and matching human performance.

Contribution

The paper proposes a novel forward-inverse modeling approach for intention understanding, effectively decomposing the task into goal generation and structured inference.

Findings

Achieves up to 80% accuracy on IntentQA and Inst-IT Bench datasets.

Surpasses baseline performance by 30%.

Matches human-level intention recognition performance.

Abstract

Improving the effectiveness of human-robot interaction requires social robots to accurately infer human goals through robust intention understanding. This challenge is particularly critical in multimodal settings, where agents must integrate heterogeneous signals including text, visual cues to form a coherent interpretation of user intent. This paper presents IntentVLM, a novel two-stage video-language framework designed for open-vocabulary human intention recognition. The approach is inspired by forward-inverse modeling in cognitive science by decomposing intention understanding into goal candidate generation followed by structured inference through selection, effectively reducing hallucinations in latent reasoning. Evaluated on the IntentQA and Inst-IT Bench datasets, IntentVLM achieves state-of-the-art results with up to 80% accuracy, notably surpassing the baseline performance by 30% and matches human performance. Our findings demonstrate that this structured reasoning approach enhances open-vocabulary intention understanding without catastrophic forgetting, offering a robust foundation for human-centered robotics.