Visualizing Semantic Structures of Sequential Data by Learning Temporal Dependencies

TL;DR

This paper introduces Temporal Dependency Network (TDN), a novel method that captures and visualizes variable-length semantic dependencies in sequential data like videos, improving interpretability of temporal structures.

Contribution

The paper proposes TDN, a new graph-based approach that learns and visualizes semantic temporal dependencies in sequential data, extending beyond fixed-length interactions.

Findings

Successfully visualized semantic structures in videos

Discovered variable-length temporal dependencies

Enhanced interpretability of sequential data models

Abstract

While conventional methods for sequential learning focus on interaction between consecutive inputs, we suggest a new method which captures composite semantic flows with variable-length dependencies. In addition, the semantic structures within given sequential data can be interpreted by visualizing temporal dependencies learned from the method. The proposed method, called Temporal Dependency Network (TDN), represents a video as a temporal graph whose node represents a frame of the video and whose edge represents the temporal dependency between two frames of a variable distance. The temporal dependency structure of semantic is discovered by learning parameterized kernels of graph convolutional methods. We evaluate the proposed method on the large-scale video dataset, Youtube-8M. By visualizing the temporal dependency structures as experimental results, we show that the suggested method can find the temporal dependency structures of video semantic.