ETRI-Activity3D: A Large-Scale RGB-D Dataset for Robots to Recognize Daily Activities of the Elderly

TL;DR

This paper introduces ETRI-Activity3D, a large-scale dataset of elderly daily activities for robot recognition, and proposes a novel four-stream adaptive CNN for improved activity classification.

Contribution

The paper presents a new large-scale dataset tailored for elderly activity recognition and introduces FSA-CNN, a robust, adaptive network architecture for analyzing multimodal activity data.

Findings

FSA-CNN outperforms existing models on NTU RGB+D and ETRI-Activity3D datasets.

The dataset captures realistic elderly activities with diverse modalities.

Domain differences between age groups affect recognition performance.

Abstract

Deep learning, based on which many modern algorithms operate, is well known to be data-hungry. In particular, the datasets appropriate for the intended application are difficult to obtain. To cope with this situation, we introduce a new dataset called ETRI-Activity3D, focusing on the daily activities of the elderly in robot-view. The major characteristics of the new dataset are as follows: 1) practical action categories that are selected from the close observation of the daily lives of the elderly; 2) realistic data collection, which reflects the robot's working environment and service situations; and 3) a large-scale dataset that overcomes the limitations of the current 3D activity analysis benchmark datasets. The proposed dataset contains 112,620 samples including RGB videos, depth maps, and skeleton sequences. During the data acquisition, 100 subjects were asked to perform 55 daily activities. Additionally, we propose a novel network called four-stream adaptive CNN (FSA-CNN). The proposed FSA-CNN has three main properties: robustness to spatio-temporal variations, input-adaptive activation function, and extension of the conventional two-stream approach. In the experiment section, we confirmed the superiority of the proposed FSA-CNN using NTU RGB+D and ETRI-Activity3D. Further, the domain difference between both groups of age was verified experimentally. Finally, the extension of FSA-CNN to deal with the multimodal data was investigated.