DTR: A Unified Deep Tensor Representation Framework for Multimedia Data Recovery

TL;DR

This paper introduces DTR, a deep tensor representation framework that combines deep generative and transform modules to improve multimedia data recovery, outperforming existing methods especially in fine detail preservation.

Contribution

The paper presents a novel unified deep tensor representation framework that enhances characterization of tensor data, advancing beyond shallow matrix factorization techniques.

Findings

DTR achieves superior recovery performance over state-of-the-art methods.

DTR effectively preserves fine details in multimedia data.

Extensive experiments validate the effectiveness of DTR.

Abstract

Recently, the transform-based tensor representation has attracted increasing attention in multimedia data (e.g., images and videos) recovery problems, which consists of two indispensable components, i.e., transform and characterization. Previously, the development of transform-based tensor representation mainly focuses on the transform aspect. Although several attempts consider using shallow matrix factorization (e.g., singular value decomposition and negative matrix factorization) to characterize the frontal slices of transformed tensor (termed as latent tensor), the faithful characterization aspect is underexplored. To address this issue, we propose a unified Deep Tensor Representation (termed as DTR) framework by synergistically combining the deep latent generative module and the deep transform module. Especially, the deep latent generative module can faithfully generate the latent tensor as compared with shallow matrix factorization. The new DTR framework not only allows us to better understand the classic shallow representations, but also leads us to explore new representation. To examine the representation ability of the proposed DTR, we consider the representative multi-dimensional data recovery task and suggest an unsupervised DTR-based multi-dimensional data recovery model. Extensive experiments demonstrate that DTR achieves superior performance compared to state-of-the-art methods in both quantitative and qualitative aspects, especially for fine details recovery.