GRACE: Generating Socially Appropriate Robot Actions Leveraging LLMs and Human Explanations

TL;DR

GRACE is a novel method that combines large language models and human explanations to enable robots to generate socially appropriate actions and explanations, improving robot-human interaction in social contexts.

Contribution

The paper introduces GRACE, a new approach that integrates common sense knowledge from LLMs with human explanations for socially aware robot action generation.

Findings

Outperforms baseline methods in generating socially appropriate actions

Enhances robot understanding through integration of human explanations

Provides sensible explanations for robot actions

Abstract

When operating in human environments, robots need to handle complex tasks while both adhering to social norms and accommodating individual preferences. For instance, based on common sense knowledge, a household robot can predict that it should avoid vacuuming during a social gathering, but it may still be uncertain whether it should vacuum before or after having guests. In such cases, integrating common-sense knowledge with human preferences, often conveyed through human explanations, is fundamental yet a challenge for existing systems. In this paper, we introduce GRACE, a novel approach addressing this while generating socially appropriate robot actions. GRACE leverages common sense knowledge from LLMs, and it integrates this knowledge with human explanations through a generative network. The bidirectional structure of GRACE enables robots to refine and enhance LLM predictions by utilizing human explanations and makes robots capable of generating such explanations for human-specified actions. Our evaluations show that integrating human explanations boosts GRACE's performance, where it outperforms several baselines and provides sensible explanations.