Efficient forward propagation of time-sequences in convolutional neural networks using Deep Shifting

TL;DR

Deep Shifting is a method that reduces redundant computations in convolutional neural networks processing continuous time-sequences, significantly saving time especially with many time-frames.

Contribution

The paper introduces Deep Shifting, a novel approach that reuses previous convolution results to minimize calculations in real-time sequence processing.

Findings

Reduces computational complexity at least to a constant, potentially quadratic.

Demonstrates significant computation time savings in practical implementations.

Effective for networks processing large numbers of time-frames.

Abstract

When a Convolutional Neural Network is used for on-the-fly evaluation of continuously updating time-sequences, many redundant convolution operations are performed. We propose the method of Deep Shifting, which remembers previously calculated results of convolution operations in order to minimize the number of calculations. The reduction in complexity is at least a constant and in the best case quadratic. We demonstrate that this method does indeed save significant computation time in a practical implementation, especially when the networks receives a large number of time-frames.