Unsupervised categorization of similarity measures

TL;DR

This paper introduces a neural network-based method for unsupervised categorization of metric spaces representing object features, overcoming limitations of traditional axis independence constraints to better differentiate features like color and shape.

Contribution

The authors propose a novel approach constraining only the metric spaces to be independent, not the axes, enabling more natural categorization of high-dimensional feature spaces.

Findings

Neural networks can autonomously categorize metric spaces via representation learning.

Mutually independent axes are insufficient for categorizing feature spaces.

The new method improves the mathematical understanding of neural network differentiation.

Abstract

In general, objects can be distinguished on the basis of their features, such as color or shape. In particular, it is assumed that similarity judgments about such features can be processed independently in different metric spaces. However, the unsupervised categorization mechanism of metric spaces corresponding to object features remains unknown. Here, we show that the artificial neural network system can autonomously categorize metric spaces through representation learning to satisfy the algebraic independence between neural networks, and project sensory information onto multiple high-dimensional metric spaces to independently evaluate the differences and similarities between features. Conventional methods often constrain the axes of the latent space to be mutually independent or orthogonal. However, the independent axes are not suitable for categorizing metric spaces. High-dimensional metric spaces that are independent of each other are not uniquely determined by the mutually independent axes, because any combination of independent axes can form mutually independent spaces. In other words, the mutually independent axes cannot be used to naturally categorize different feature spaces, such as color space and shape space. Therefore, constraining the axes to be mutually independent makes it difficult to categorize high-dimensional metric spaces. To overcome this problem, we developed a method to constrain only the spaces to be mutually independent and not the composed axes to be independent. Our theory provides general conditions for the unsupervised categorization of independent metric spaces, thus advancing the mathematical theory of functional differentiation of neural networks.