Analysis of Latent-Space Motion for Collaborative Intelligence

TL;DR

This paper investigates how motion in input videos translates to feature tensors within deep neural networks, revealing that each channel's motion approximates scaled input motion, aiding in compression and analysis for collaborative intelligence.

Contribution

It provides a theoretical analysis linking input motion to feature tensor motion in DNNs, validated by experiments, facilitating improved processing in collaborative intelligence.

Findings

Feature tensor motion closely relates to scaled input motion.

Analysis validated with common motion models.

Insights support better compression and analysis of feature sequences.

Abstract

When the input to a deep neural network (DNN) is a video signal, a sequence of feature tensors is produced at the intermediate layers of the model. If neighboring frames of the input video are related through motion, a natural question is, "what is the relationship between the corresponding feature tensors?" By analyzing the effect of common DNN operations on optical flow, we show that the motion present in each channel of a feature tensor is approximately equal to the scaled version of the input motion. The analysis is validated through experiments utilizing common motion models. %These results will be useful in collaborative intelligence applications where sequences of feature tensors need to be compressed or further analyzed.