Recognizing Dynamic Scenes with Deep Dual Descriptor based on Key Frames and Key Segments

TL;DR

This paper introduces a novel deep dual descriptor method for recognizing dynamic scenes by selecting key frames and segments, effectively capturing static and dynamic features for improved scene understanding.

Contribution

It proposes a new approach using key frames and segments with deep neural networks for dynamic scene recognition, achieving state-of-the-art results.

Findings

Achieved state-of-the-art performance on public datasets.

Effectively captures static and dynamic scene features.

Demonstrated robustness across multiple dynamic scene classes.

Abstract

Recognizing dynamic scenes is one of the fundamental problems in scene understanding, which categorizes moving scenes such as a forest fire, landslide, or avalanche. While existing methods focus on reliable capturing of static and dynamic information, few works have explored frame selection from a dynamic scene sequence. In this paper, we propose dynamic scene recognition using a deep dual descriptor based on `key frames' and `key segments.' Key frames that reflect the feature distribution of the sequence with a small number are used for capturing salient static appearances. Key segments, which are captured from the area around each key frame, provide an additional discriminative power by dynamic patterns within short time intervals. To this end, two types of transferred convolutional neural network features are used in our approach. A fully connected layer is used to select the key frames and key segments, while the convolutional layer is used to describe them. We conducted experiments using public datasets as well as a new dataset comprised of 23 dynamic scene classes with 10 videos per class. The evaluation results demonstrated the state-of-the-art performance of the proposed method.