Very Basics of Tensors with Graphical Notations: Unfolding, Calculations, and Decompositions

TL;DR

This paper introduces the fundamental concepts of tensors and their graphical notation, illustrating how various tensor operations can be visually represented to aid understanding in signal processing and machine learning.

Contribution

It provides a clear, beginner-friendly explanation of tensor basics and graphical notation, filling a gap often overlooked in tensor-related literature.

Findings

Graphical notation simplifies tensor multiplication understanding

Most tensor products can be represented visually

The note aids beginners in grasping tensor operations

Abstract

Tensor network diagram (graphical notation) is a useful tool that graphically represents multiplications between multiple tensors using nodes and edges. Using the graphical notation, complex multiplications between tensors can be described simply and intuitively, and it also helps to understand the essence of tensor products. In fact, most of matrix/tensor products including inner product, outer product, Hadamard product, Kronecker product, and Khatri-Rao product can be written in graphical notation. These matrix/tensor operations are essential building blocks for the use of matrix/tensor decompositions in signal processing and machine learning. The purpose of this lecture note is to learn the very basics of tensors and how to represent them in mathematical symbols and graphical notation. Many papers using tensors omit these detailed definitions and explanations, which can be difficult for the reader. I hope this note will be of help to such readers.