Information Mandala: Statistical Distance Matrix with Clustering

TL;DR

This paper introduces the Information Mandala, a matrix form of statistical distance that enhances the representation of feature dissimilarities, aiding in object recognition and understanding CNN principles.

Contribution

It extends traditional scalar statistical distance to a matrix form and demonstrates its effectiveness in visualizing and clustering image features.

Findings

Effective in object recognition tasks

Visualizes dissimilarities in CIFAR images

Reveals geometric clustering patterns

Abstract

In machine learning, observation features are measured in a metric space to obtain their distance function for optimization. Given similar features that are statistically sufficient as a population, a statistical distance between two probability distributions can be calculated for more precise learning. Provided the observed features are multi-valued, the statistical distance function is still efficient. However, due to its scalar output, it cannot be applied to represent detailed distances between feature elements. To resolve this problem, this paper extends the traditional statistical distance to a matrix form, called a statistical distance matrix. In experiments, the proposed approach performs well in object recognition tasks and clearly and intuitively represents the dissimilarities between cat and dog images in the CIFAR dataset, even when directly calculated using the image pixels. By using the hierarchical clustering of the statistical distance matrix, the image pixels can be separated into several clusters that are geometrically arranged around a center like a Mandala pattern. The statistical distance matrix with clustering, called the Information Mandala, is beyond ordinary saliency maps and can help to understand the basic principles of the convolution neural network.