Continuous Histogram Loss: Beyond Neural Similarity

TL;DR

This paper introduces Continuous Histogram Loss (CHL), a novel differentiable loss function that generalizes histogram loss to handle continuous similarity values, enabling broader applications in similarity and representation learning.

Contribution

The paper proposes CHL, a new loss function that extends histogram loss to continuous similarities, allowing for more flexible and effective similarity learning tasks.

Findings

CHL effectively models continuous similarity distributions.

CHL improves performance in similarity and representation learning tasks.

CHL is applicable to data visualization and other related tasks.

Abstract

Similarity learning has gained a lot of attention from researches in recent years and tons of successful approaches have been recently proposed. However, the majority of the state-of-the-art similarity learning methods consider only a binary similarity. In this paper we introduce a new loss function called Continuous Histogram Loss (CHL) which generalizes recently proposed Histogram loss to multiple-valued similarities, i.e. allowing the acceptable values of similarity to be continuously distributed within some range. The novel loss function is computed by aggregating pairwise distances and similarities into 2D histograms in a differentiable manner and then computing the probability of condition that pairwise distances will not decrease as the similarities increase. The novel loss is capable of solving a wider range of tasks including similarity learning, representation learning and data visualization.