An Efficient Framework for Few-shot Skeleton-based Temporal Action Segmentation

TL;DR

This paper introduces an efficient framework for few-shot skeleton-based temporal action segmentation, combining data augmentation via motion interpolation and a CTC-enhanced model to improve segmentation accuracy with limited data.

Contribution

The study presents a novel data augmentation method and integrates CTC with skeleton-based TAS models, advancing few-shot action segmentation performance.

Findings

Data augmentation significantly increases training samples.

CTC integration improves temporal alignment and segmentation accuracy.

Framework outperforms existing methods on multiple datasets.

Abstract

Temporal action segmentation (TAS) aims to classify and locate actions in the long untrimmed action sequence. With the success of deep learning, many deep models for action segmentation have emerged. However, few-shot TAS is still a challenging problem. This study proposes an efficient framework for the few-shot skeleton-based TAS, including a data augmentation method and an improved model. The data augmentation approach based on motion interpolation is presented here to solve the problem of insufficient data, and can increase the number of samples significantly by synthesizing action sequences. Besides, we concatenate a Connectionist Temporal Classification (CTC) layer with a network designed for skeleton-based TAS to obtain an optimized model. Leveraging CTC can enhance the temporal alignment between prediction and ground truth and further improve the segment-wise metrics of segmentation results. Extensive experiments on both public and self-constructed datasets, including two small-scale datasets and one large-scale dataset, show the effectiveness of two proposed methods in improving the performance of the few-shot skeleton-based TAS task.