Tensor Methods in Computer Vision and Deep Learning

TL;DR

This paper reviews the role of tensor methods in computer vision and deep learning, highlighting their applications in data representation, architecture design, robustness, and theoretical understanding, supported by practical Python examples.

Contribution

It provides a comprehensive review of tensor techniques in deep learning for vision, including recent advances and practical implementations with TensorLy.

Findings

Tensor methods enhance visual data analysis and deep learning architectures.

They improve robustness to noise and adversarial attacks.

Tensor techniques facilitate theoretical insights into deep networks.

Abstract

Tensors, or multidimensional arrays, are data structures that can naturally represent visual data of multiple dimensions. Inherently able to efficiently capture structured, latent semantic spaces and high-order interactions, tensors have a long history of applications in a wide span of computer vision problems. With the advent of the deep learning paradigm shift in computer vision, tensors have become even more fundamental. Indeed, essential ingredients in modern deep learning architectures, such as convolutions and attention mechanisms, can readily be considered as tensor mappings. In effect, tensor methods are increasingly finding significant applications in deep learning, including the design of memory and compute efficient network architectures, improving robustness to random noise and adversarial attacks, and aiding the theoretical understanding of deep networks. This article provides an in-depth and practical review of tensors and tensor methods in the context of representation learning and deep learning, with a particular focus on visual data analysis and computer vision applications. Concretely, besides fundamental work in tensor-based visual data analysis methods, we focus on recent developments that have brought on a gradual increase of tensor methods, especially in deep learning architectures, and their implications in computer vision applications. To further enable the newcomer to grasp such concepts quickly, we provide companion Python notebooks, covering key aspects of the paper and implementing them, step-by-step with TensorLy.