A Systematic Review on Reproducibility in Child-Robot Interaction

TL;DR

This systematic review analyzes recent child-robot interaction research for reproducibility issues, highlighting reporting deficiencies and proposing guidelines to enhance transparency and replicability in the field.

Contribution

It provides a comprehensive analysis of reproducibility challenges in CRI and offers a practical checklist to improve reporting standards.

Findings

Significant reporting deficiencies in demographics and study design.

Lack of open data and active code repositories.

Guidelines and checklists proposed to improve reproducibility.

Abstract

Research reproducibility - i.e., rerunning analyses on original data to replicate the results - is paramount for guaranteeing scientific validity. However, reproducibility is often very challenging, especially in research fields where multi-disciplinary teams are involved, such as child-robot interaction (CRI). This paper presents a systematic review of the last three years (2020-2022) of research in CRI under the lens of reproducibility, by analysing the field for transparency in reporting. Across a total of 325 studies, we found deficiencies in reporting demographics (e.g. age of participants), study design and implementation (e.g. length of interactions), and open data (e.g. maintaining an active code repository). From this analysis, we distill a set of guidelines and provide a checklist to systematically report CRI studies to help and guide research to improve reproducibility in CRI and beyond.