OREBA: A Dataset for Objectively Recognizing Eating Behaviour and Associated Intake

TL;DR

This paper introduces the OREBA dataset, a comprehensive multi-sensor collection of eating behavior data during communal meals, enabling improved automatic detection of intake gestures with deep learning models.

Contribution

The paper presents the creation and detailed description of the OREBA dataset, including sensor data, annotation, and baseline deep learning results for intake gesture detection.

Findings

Deep learning models achieved F1 scores around 0.85 for intake gesture detection.

The dataset includes synchronized video and IMU data from 202 participants during communal meals.

Baseline results provide a benchmark for future research in dietary monitoring.

Abstract

Automatic detection of intake gestures is a key element of automatic dietary monitoring. Several types of sensors, including inertial measurement units (IMU) and video cameras, have been used for this purpose. The common machine learning approaches make use of the labeled sensor data to automatically learn how to make detections. One characteristic, especially for deep learning models, is the need for large datasets. To meet this need, we collected the Objectively Recognizing Eating Behavior and Associated Intake (OREBA) dataset. The OREBA dataset aims to provide comprehensive multi-sensor data recorded during the course of communal meals for researchers interested in intake gesture detection. Two scenarios are included, with 100 participants for a discrete dish and 102 participants for a shared dish, totalling 9069 intake gestures. Available sensor data consists of synchronized frontal video and IMU with accelerometer and gyroscope for both hands. We report the details of data collection and annotation, as well as details of sensor processing. The results of studies on IMU and video data involving deep learning models are reported to provide a baseline for future research. Specifically, the best baseline models achieve performances of $F_1$ = 0.853 for the discrete dish using video and $F_1$ = 0.852 for the shared dish using inertial data.