Cabbage Sweeter than Cake? Analysing the Potential of Large Language Models for Learning Conceptual Spaces

TL;DR

This paper investigates the capacity of large language models to learn conceptual spaces, finding that fine-tuned BERT models can perform comparably or better than larger GPT-3 models in representing perceptual features.

Contribution

It demonstrates that smaller, fine-tuned BERT models can effectively learn conceptual spaces, challenging the assumption that larger models are always superior for this task.

Findings

Fine-tuned BERT models match or outperform GPT-3 in learning conceptual spaces.

LLMs can learn meaningful perceptual representations to some extent.

Smaller models can be more effective than larger ones when fine-tuned appropriately.

Abstract

The theory of Conceptual Spaces is an influential cognitive-linguistic framework for representing the meaning of concepts. Conceptual spaces are constructed from a set of quality dimensions, which essentially correspond to primitive perceptual features (e.g. hue or size). These quality dimensions are usually learned from human judgements, which means that applications of conceptual spaces tend to be limited to narrow domains (e.g. modelling colour or taste). Encouraged by recent findings about the ability of Large Language Models (LLMs) to learn perceptually grounded representations, we explore the potential of such models for learning conceptual spaces. Our experiments show that LLMs can indeed be used for learning meaningful representations to some extent. However, we also find that fine-tuned models of the BERT family are able to match or even outperform the largest GPT-3 model, despite being 2 to 3 orders of magnitude smaller.