A Platform to Collect, Unify, and Distribute Inertial Labeled Signals for Human Activity Recognition

TL;DR

This paper presents a platform designed to collect, unify, and distribute large, homogeneous inertial signal datasets to enhance the training of deep learning models for human activity recognition.

Contribution

It introduces a novel platform architecture that supports long-term data collection and integration of inertial datasets for HAR research.

Findings

Platform architecture has been defined and main components developed.

The platform facilitates large-scale, homogeneous data collection.

Initial verification shows the approach is sound.

Abstract

Human activity recognition (HAR) is a very active research field. Recently, deep learning techniques are being exploited to recognize human activities from inertial signals. However, to compute accurate and reliable deep learning models, a huge amount of data is required. The goal of our work is the definition of a platform to support long-term data collection to be used in training of HAR algorithms. The platform aims to integrate datasets of inertial signals in order to make available to the scientific community a large dataset of homogeneous data. The architecture of the platform has been defined and some of the main components have been developed in order to verify the soundness of the approach.

Figures

Full text

I Introduction

Recognition of activities of daily living (ADL) from inertial signals is a very active research field in view of the many application domains interested in (e.g., sports [1] and healthcare [2]) and the increasing diffusion of wearable devices embodying inertial sensors. However, the classification of sensor data represents the main challenge of human activity recognition (HAR) because the space of the signals are not perfectly separated.

Preliminary HAR techniques exploited supervised machine learning algorithms. The main challenges of these techniques include: the difficulty of transferring the performances achieved in laboratory to a real context [3], and the inability of the algorithms to extract and organize discriminative information from the data [4].

In recent years, deep learning has been successfully applied to 3D and 4D signals, and more recently it has been exploited also for 1D signals [5, 6]. The widespread use of deep learning techniques is justified by their properties of local dependency and scale invariance [7]. Furthermore, deep learning methodologies permit automated discovery of abstraction which overcomes the features extraction issue [4]. While deep learning techniques are powerful and achieve high performance, they rely on very complex models that strictly depend on the estimation of a large number of parameters, which requires a considerable amount of available data [8].

In recent years, some researchers have published their own dataset related to HAR. However, these datasets are heterogeneous and their standardization in a single unified dataset requires considerable effort. For example, signals are expressed in different units of measure, they may include gravity or not, and signals have different acquisition frequencies. Furthermore, labels are not aligned with a common dictionary and sometimes have different meanings among different datasets (’sitting’ may refer to the state of being seated in a chair or the transition from standing to sitting).

Since acquiring labeled time series is a costly procedure in terms of resource, time, and people involved, we think that integrating existing datasets is the right direction despite the strong heterogeneity of the data. This abstract presents a platform that semi-automatically integrates heterogeneous data and provides them in a homogenous form.

II Continuous Learning Platform

The main aim of the Continuous Learning Platform (CLP in the sequel) is to make available (i) a large amount of labeled inertial signals related of ADLs and falls, (ii) a catalogue of downloadable activity recognition models, and (iii) a service that, given a set of raw data, identifies the corresponding ADL.

CLP collects inertial signals from existing datasets or applications, manages the collected inertial signals, and distributes uniformed labeled inertial signals, activity recognition models, and labels assigned to series of inertial signals (Figure 1).

II-A Data Collection

The Data Collection component acquires existing inertial signals and uniforms their representations. Existing inertial signals can be from i) existing datasets (e.g., UniMiB SHAR [9]); ii) labeled inertial signals from applications used by volunteers to record datasets (e.g., UniMiB AAL [10]); and iii) non-labeled inertial signals from applications used by people performing ADLs (e.g., Sensor Data Logger [11]).

One of the main issues in handling multiple datasets is the lack of consistency in terms of how the data is stored and what are the information (e.g., in the UCI HAR dataset [12] data are organized in txt files stored in 2 directories; in MobiAct [13] data are organized in csv files stored in 20 directories). In Continuous Learning Platform we enforce a single storage technology for all data and a single structure for the data.

The Data Collection component includes the Driver Loader, the Dataset Loader, and the Importer modules, which respectively allow to load custom drivers developed to support specific datasets, to load datasets to be integrated in Unified Dataset, and to ask for the integration of the new datasets into the Unified Dataset. Separating the Dataset Loader from the Driver Loader, allows the dynamic on-boarding of the driver, which may require a reboot of the Dataset Loader service in order to be visible and exploitable from the service itself.

II-B Data Management

The Data Management component integrates the new labelled signals into the Unified Dataset and makes available sets of labelled signals to those who need them (Data Access). Before being included into the Unified Dataset, signals require to be homogeneous both in terms of representation and label. Homogeneous representations of the input data are required by machine learning methods [14]. Labels must be unique for the same activity and the same labels must be assigned to the same signals, otherwise the classifiers will not work efficiently.

The Data Management component includes the following modules. The Data Aligner module pre-processes the data from the Data Collection component in order to make them usable by any machine learning method. For example, an activity that is in charge of the Data Aligner module is the conversion to a same measurement unit. The Label Consolidator module is in charge of uniforming the labels of the dataset to include to a common unified set. For example, if a dataset uses the label ’sitting’ to label signals related to the transition (from standing to sitting down) and in the Unified Dataset is used ’sit down’ to label signals related to the transition (from standing to sitting down), then the label will be changed to be consistent with the Unified Dataset. In view of the delicate nature of this procedure, this module is intended to be semi-automatic: it provides suggestions on the assignment of labels, but ultimately it is down to the end user to decide whether or not to accept the suggestions. In terms of data distribution, the Data Composer module simply intercepts requests for sets of labeled signals. For example, a request can be: “all signals labeled running”.

II-C Data Distribution

The Data Distribution component role is to provide i) sets of labeled signals according to specific needs; ii) trained classifiers; and iii) labels corresponding to the activities performed given frames of signals.

The Data Distribution component includes the following modules. The Classifier Builder module is in charge of distributing activity recognition models that can be integrated in domain dependant applications. The module also relies on the Classifier Deployer to store the new trained activity recognition model to be used for online classification. Finally, the Online Classifier module provides online services related to classification: given a set of inertial signals, it provides information regarding the activity the subject is performing.

III Conclusions

The lack of large datasets penalizes the possibility of exploiting deep learning techniques for human activity recognition.

We designed a platform which integrates data from heterogenous sources and provides several types of access to the unified data.

The framework has been partially implemented. We have prioritized the development of the most challenging components: Data Collection and Data Management [15]. The modules are web services providing the services through RESTFul APIs. Till now, five datasets have been integrated.