Overview of Dialogue Robot Competition 2023

TL;DR

The paper summarizes the 2023 Dialogue Robot Competition, highlighting its new challenging task requiring real-time situational information handling and providing preliminary results and evaluation methods.

Contribution

It introduces a more complex dialogue task for robots involving real-time information, expanding on previous competitions and setting new benchmarks.

Findings

Preliminary round held at real travel agency stores

Participants developed systems utilizing real-time situational data

Evaluation methods for the competition are outlined

Abstract

We have held dialogue robot competitions in 2020 and 2022 to compare the performances of interactive robots using an android that closely resembles a human. In 2023, the third competition DRC2023 was held. The task of DRC2023 was designed to be more challenging than the previous travel agent dialogue tasks. Since anyone can now develop a dialogue system using LLMs, the participating teams are required to develop a system that effectively uses information about the situation on the spot (real-time information), which is not handled by ChatGPT and other systems. DRC2023 has two rounds, a preliminary round and the final round as well as the previous competitions. The preliminary round has held on Oct.27 -- Nov.20, 2023 at real travel agency stores. The final round will be held on December 23, 2023. This paper provides an overview of the task settings and evaluation method of DRC2023 and the preliminary round results.