Distance Matters in Human-Object Interaction Detection

TL;DR

This paper introduces a novel two-stage approach with a distance-aware attention module and loss function to improve human-object interaction detection, especially for distant interactions, achieving state-of-the-art results.

Contribution

It proposes a new distance-aware attention mechanism and loss function specifically designed to enhance detection of distant human-object interactions.

Findings

Significant performance improvement on HICO-DET and V-COCO datasets.

Outperforms existing methods by a large margin.

Achieves new state-of-the-art results in HOI detection.

Abstract

Human-Object Interaction (HOI) detection has received considerable attention in the context of scene understanding. Despite the growing progress on benchmarks, we realize that existing methods often perform unsatisfactorily on distant interactions, where the leading causes are two-fold: 1) Distant interactions are by nature more difficult to recognize than close ones. A natural scene often involves multiple humans and objects with intricate spatial relations, making the interaction recognition for distant human-object largely affected by complex visual context. 2) Insufficient number of distant interactions in benchmark datasets results in under-fitting on these instances. To address these problems, in this paper, we propose a novel two-stage method for better handling distant interactions in HOI detection. One essential component in our method is a novel Far Near Distance Attention module. It enables information propagation between humans and objects, whereby the spatial distance is skillfully taken into consideration. Besides, we devise a novel Distance-Aware loss function which leads the model to focus more on distant yet rare interactions. We conduct extensive experiments on two challenging datasets - HICO-DET and V-COCO. The results demonstrate that the proposed method can surpass existing approaches by a large margin, resulting in new state-of-the-art performance.