A Baseline Study and Benchmark for Few-Shot Open-Set Action Recognition with Feature Residual Discrimination

TL;DR

This paper introduces a new feature residual discriminator method for few-shot open-set action recognition in videos, significantly improving unknown action rejection while maintaining accuracy, and establishes a new benchmark in the field.

Contribution

It proposes an architectural extension with a Feature-Residual Discriminator for FSOS-AR, advancing open-set recognition in complex video data.

Findings

FR-Disc significantly improves unknown rejection

Maintains closed-set accuracy

Sets new state-of-the-art for FSOS-AR

Abstract

Few-Shot Action Recognition (FS-AR) has shown promising results but is often limited by a closed-set assumption that fails in real-world open-set scenarios. While Few-Shot Open-Set (FSOS) recognition is well-established for images, its extension to spatio-temporal video data remains underexplored. To address this, we propose an architectural extension based on a Feature-Residual Discriminator (FR-Disc), adapting previous work on skeletal data to the more complex video domain. Extensive experiments on five datasets demonstrate that while common open-set techniques provide only marginal gains, our FR-Disc significantly enhances unknown rejection capabilities without compromising closed-set accuracy, setting a new state-of-the-art for FSOS-AR. The project website, code, and benchmark are available at: https://hsp-iit.github.io/fsosar/.