Subjects and Their Objects: Localizing Interactees for a Person-Centric View of Importance
Chao-Yeh Chen, Kristen Grauman

TL;DR
This paper introduces a method to localize the object of a person's action in images by predicting the likely position and scale of interactees, independent of specific actions, enhancing understanding of image context.
Contribution
It proposes a novel, action-independent approach to predict interactee locations using pose, gaze, and scene cues, and introduces a new dataset for evaluation.
Findings
Effective in localizing interactees across diverse images
Improves performance in object detection and scene understanding tasks
Demonstrates utility in natural language scene description
Abstract
Understanding images with people often entails understanding their \emph{interactions} with other objects or people. As such, given a novel image, a vision system ought to infer which other objects/people play an important role in a given person's activity. However, existing methods are limited to learning action-specific interactions (e.g., how the pose of a tennis player relates to the position of his racquet when serving the ball) for improved recognition, making them unequipped to reason about novel interactions with actions or objects unobserved in the training data. We propose to predict the "interactee" in novel images---that is, to localize the \emph{object} of a person's action. Given an arbitrary image with a detected person, the goal is to produce a saliency map indicating the most likely positions and scales where that person's interactee would be found. To that end, we…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVisual Attention and Saliency Detection · Multimodal Machine Learning Applications · Advanced Neural Network Applications
