Relational Future Captioning Model for Explaining Likely Collisions in Daily Tasks

TL;DR

This paper introduces the Relational Future Captioning Model (RFCM), a novel approach for generating explanatory captions about potential future collisions in domestic robot tasks, enhancing robot understanding and safety.

Contribution

The paper presents RFCM with a Relational Self-Attention Encoder, improving relationship extraction in future event captioning for robots.

Findings

RFCM outperforms baseline methods on two datasets

Relational Self-Attention improves event relationship modeling

Enhanced explanation of collision risks in robot tasks

Abstract

Domestic service robots that support daily tasks are a promising solution for elderly or disabled people. It is crucial for domestic service robots to explain the collision risk before they perform actions. In this paper, our aim is to generate a caption about a future event. We propose the Relational Future Captioning Model (RFCM), a crossmodal language generation model for the future captioning task. The RFCM has the Relational Self-Attention Encoder to extract the relationships between events more effectively than the conventional self-attention in transformers. We conducted comparison experiments, and the results show the RFCM outperforms a baseline method on two datasets.