HOID-R1: Reinforcement Learning for Open-World Human-Object Interaction Detection Reasoning with Multimodal Large Language Model

TL;DR

HOID-R1 is a novel reinforcement learning framework that enhances open-world human-object interaction detection by integrating multimodal large language models with reasoning supervision and policy optimization.

Contribution

It introduces a new HOI detection method combining chain-of-thought guided fine-tuning and group relative policy optimization within a reinforcement learning framework.

Findings

Achieves state-of-the-art results on HOI detection benchmarks.

Outperforms existing methods in open-world generalization.

Reduces hallucinations in reasoning outputs through an MLLM-as-a-judge mechanism.

Abstract

Understanding and recognizing human-object interaction (HOI) is a pivotal application in AR/VR and robotics. Recent open-vocabulary HOI detection approaches depend exclusively on large language models for richer textual prompts, neglecting their inherent 3D spatial understanding capabilities. To address this shortcoming, we introduce HOID-R1, the first HOI detection framework that integrates chain-of-thought (CoT) guided supervised fine-tuning (SFT) with group relative policy optimization (GRPO) within a reinforcement learning (RL) paradigm. Specifically, we initially apply SFT to imbue the model with essential reasoning capabilities, forcing the model to articulate its thought process in the output. Subsequently, we integrate GRPO to leverage multi-reward signals for policy optimization, thereby enhancing alignment across diverse modalities. To mitigate hallucinations in the CoT reasoning, we introduce an "MLLM-as-a-judge" mechanism that supervises the CoT outputs, further improving generalization. Extensive experiments show that HOID-R1 achieves state-of-the-art performance on HOI detection benchmarks and outperforms existing methods in open-world generalization to novel scenarios.