Understanding Human Activity with Uncertainty Measure for Novelty in Graph Convolutional Networks

TL;DR

This paper introduces a Temporal Fusion Graph Convolutional Network with uncertainty measures to improve human activity recognition, reduce over-segmentation, and better handle novel or unforeseen scenarios in human-robot collaboration.

Contribution

It proposes a novel network architecture combined with spectral normalization and Gaussian process-based uncertainty estimation to enhance activity boundary detection and novelty recognition.

Findings

Improved boundary estimation reduces over-segmentation.

Enhanced uncertainty quantification detects novel activities.

Robust handling of unforeseen scenarios in activity recognition.

Abstract

Understanding human activity is a crucial aspect of developing intelligent robots, particularly in the domain of human-robot collaboration. Nevertheless, existing systems encounter challenges such as over-segmentation, attributed to errors in the up-sampling process of the decoder. In response, we introduce a promising solution: the Temporal Fusion Graph Convolutional Network. This innovative approach aims to rectify the inadequate boundary estimation of individual actions within an activity stream and mitigate the issue of over-segmentation in the temporal dimension. Moreover, systems leveraging human activity recognition frameworks for decision-making necessitate more than just the identification of actions. They require a confidence value indicative of the certainty regarding the correspondence between observations and training examples. This is crucial to prevent overly confident responses to unforeseen scenarios that were not part of the training data and may have resulted in mismatches due to weak similarity measures within the system. To address this, we propose the incorporation of a Spectral Normalized Residual connection aimed at enhancing efficient estimation of novelty in observations. This innovative approach ensures the preservation of input distance within the feature space by imposing constraints on the maximum gradients of weight updates. By limiting these gradients, we promote a more robust handling of novel situations, thereby mitigating the risks associated with overconfidence. Our methodology involves the use of a Gaussian process to quantify the distance in feature space.