Learning Human-Aligned Representations with Contrastive Learning and Generative Similarity

TL;DR

This paper introduces a contrastive learning method that uses a Bayesian generative similarity measure to learn human-aligned representations, effectively capturing human-like concepts in visual data.

Contribution

It proposes a novel approach combining contrastive learning with Bayesian generative similarity to produce human-aligned embeddings in complex domains.

Findings

Captures human-like representations of shape regularity

Models abstract Euclidean geometric concepts

Represents semantic hierarchies in natural images

Abstract

Humans rely on effective representations to learn from few examples and abstract useful information from sensory data. Inducing such representations in machine learning models has been shown to improve their performance on various benchmarks such as few-shot learning and robustness. However, finding effective training procedures to achieve that goal can be challenging as psychologically rich training data such as human similarity judgments are expensive to scale, and Bayesian models of human inductive biases are often intractable for complex, realistic domains. Here, we address this challenge by leveraging a Bayesian notion of generative similarity whereby two data points are considered similar if they are likely to have been sampled from the same distribution. This measure can be applied to complex generative processes, including probabilistic programs. We incorporate generative similarity into a contrastive learning objective to enable learning of embeddings that express human cognitive representations. We demonstrate the utility of our approach by showing that it can be used to capture human-like representations of shape regularity, abstract Euclidean geometric concepts, and semantic hierarchies for natural images.