Can Large Language Models Create New Knowledge for Spatial Reasoning Tasks?

TL;DR

This paper investigates whether large language models can generate genuinely new spatial reasoning knowledge, demonstrating that models like Claude 3 exhibit emergent understanding beyond training data.

Contribution

It provides evidence that LLMs can perform sophisticated spatial reasoning tasks likely not encountered during training, indicating emergent knowledge capabilities.

Findings

Claude 3 performs well on spatial reasoning tasks

LLMs show signs of understanding beyond training data

Spatial reasoning ability emerges in state-of-the-art LLMs

Abstract

The potential for Large Language Models (LLMs) to generate new information offers a potential step change for research and innovation. This is challenging to assert as it can be difficult to determine what an LLM has previously seen during training, making "newness" difficult to substantiate. In this paper we observe that LLMs are able to perform sophisticated reasoning on problems with a spatial dimension, that they are unlikely to have previously directly encountered. While not perfect, this points to a significant level of understanding that state-of-the-art LLMs can now achieve, supporting the proposition that LLMs are able to yield significant emergent properties. In particular, Claude 3 is found to perform well in this regard.