Learning a perceptual manifold with deep features for animation video resequencing

TL;DR

This paper introduces a deep learning framework that generates new animation video sequences by learning a perceptual distance metric from human judgments, enabling smooth resequencing across diverse styles without manual feature engineering.

Contribution

The novel framework combines learned perceptual metrics with graph-based manifold learning for animation video resequencing, applicable to various styles and unordered image collections.

Findings

Produces smooth, visually appealing animation sequences

Applicable to a wide range of styles without manual features

Effective for arranging unordered image collections

Abstract

We propose a novel deep learning framework for animation video resequencing. Our system produces new video sequences by minimizing a perceptual distance of images from an existing animation video clip. To measure perceptual distance, we utilize the activations of convolutional neural networks and learn a perceptual distance by training these features on a small network with data comprised of human perceptual judgments. We show that with this perceptual metric and graph-based manifold learning techniques, our framework can produce new smooth and visually appealing animation video results for a variety of animation video styles. In contrast to previous work on animation video resequencing, the proposed framework applies to wide range of image styles and does not require hand-crafted feature extraction, background subtraction, or feature correspondence. In addition, we also show that our framework has applications to appealing arrange unordered collections of images.