# Unsupervised learning of action classes with continuous temporal embedding

## TL;DR

This paper introduces an unsupervised method for detecting and classifying actions in untrimmed videos by leveraging continuous temporal embeddings, eliminating the need for manual annotations.

## Contribution

It presents a novel unsupervised approach using continuous temporal embeddings to discover action classes in untrimmed videos, even with diverse and unknown content.

## Key findings

- Effective on multiple challenging datasets
- Can identify meaningful action clusters without labels
- Applicable to videos with unknown high-level activities

## Abstract

The task of temporally detecting and segmenting actions in untrimmed videos has seen an increased attention recently. One problem in this context arises from the need to define and label action boundaries to create annotations for training which is very time and cost intensive. To address this issue, we propose an unsupervised approach for learning action classes from untrimmed video sequences. To this end, we use a continuous temporal embedding of framewise features to benefit from the sequential nature of activities. Based on the latent space created by the embedding, we identify clusters of temporal segments across all videos that correspond to semantic meaningful action classes. The approach is evaluated on three challenging datasets, namely the Breakfast dataset, YouTube Instructions, and the 50Salads dataset. While previous works assumed that the videos contain the same high level activity, we furthermore show that the proposed approach can also be applied to a more general setting where the content of the videos is unknown.