Nearest Neighbor Future Captioning: Generating Descriptions for Possible Collisions in Object Placement Tasks

TL;DR

This paper introduces a novel model for future collision captioning in domestic service robots, combining nearest neighbor retrieval and collision region attention to improve descriptive accuracy of potential risks.

Contribution

It proposes the Nearest Neighbor Future Captioning Model with a Collision Attention Module, explicitly modeling collision regions for better risk description in robot planning.

Findings

Outperforms baseline methods on CIDEr-D score (33.08 vs. 25.09)

Introduces a new collision dataset for DSRs

Enhances explainability of robot collision predictions

Abstract

Domestic service robots (DSRs) that support people in everyday environments have been widely investigated. However, their ability to predict and describe future risks resulting from their own actions remains insufficient. In this study, we focus on the linguistic explainability of DSRs. Most existing methods do not explicitly model the region of possible collisions; thus, they do not properly generate descriptions of these regions. In this paper, we propose the Nearest Neighbor Future Captioning Model that introduces the Nearest Neighbor Language Model for future captioning of possible collisions, which enhances the model output with a nearest neighbors retrieval mechanism. Furthermore, we introduce the Collision Attention Module that attends regions of possible collisions, which enables our model to generate descriptions that adequately reflect the objects associated with possible collisions. To validate our method, we constructed a new dataset containing samples of collisions that can occur when a DSR places an object in a simulation environment. The experimental results demonstrated that our method outperformed baseline methods, based on the standard metrics. In particular, on CIDEr-D, the baseline method obtained 25.09 points, whereas our method obtained 33.08 points.