Learning to Sort Image Sequences via Accumulated Temporal Differences

TL;DR

This paper introduces a convolutional approach that leverages accumulated temporal differences to accurately sequence unordered images of dynamic scenes, outperforming existing methods and demonstrating strong generalization across datasets.

Contribution

The paper presents a novel convolutional block that captures spatial and temporal information through feature map differences for image sequence ordering.

Findings

Outperforms state-of-the-art methods on UCF101-based dataset

Generalizes well to DAVIS dataset without retraining

Effective in ordering unordered dynamic scene images

Abstract

Consider a set of n images of a scene with dynamic objects captured with a static or a handheld camera. Let the temporal order in which these images are captured be unknown. There can be n! possibilities for the temporal order in which these images could have been captured. In this work, we tackle the problem of temporally sequencing the unordered set of images of a dynamic scene captured with a hand-held camera. We propose a convolutional block which captures the spatial information through 2D convolution kernel and captures the temporal information by utilizing the differences present among the feature maps extracted from the input images. We evaluate the performance of the proposed approach on the dataset extracted from a standard action recognition dataset, UCF101. We show that the proposed approach outperforms the state-of-the-art methods by a significant margin. We show that the network generalizes well by evaluating it on a dataset extracted from the DAVIS dataset, a dataset meant for video object segmentation, when the same network was trained with a dataset extracted from UCF101, a dataset meant for action recognition.