Can AI Rely on the Systematicity of Truth? The Challenge of Modelling Normative Domains

TL;DR

This paper examines whether large language models can effectively model normative domains, highlighting that the asystematic nature of truth in these areas limits their ability to rely on systematicity for accurate understanding.

Contribution

It argues that the asystematicity of truth in normative domains challenges LLMs' capacity to model these areas, emphasizing the need for human agency in practical reasoning.

Findings

Normative domains exhibit largely asystematic truth.

Systematicity of truth aids LLMs in filling gaps and correcting inaccuracies.

Asystematic truth in normative domains limits LLMs' modeling capabilities.

Abstract

A key assumption fuelling optimism about the progress of large language models (LLMs) in accurately and comprehensively modelling the world is that the truth is systematic: true statements about the world form a whole that is not just consistent, in that it contains no contradictions, but coherent, in that the truths are inferentially interlinked. This holds out the prospect that LLMs might in principle rely on that systematicity to fill in gaps and correct inaccuracies in the training data: consistency and coherence promise to facilitate progress towards comprehensiveness in an LLM's representation of the world. However, philosophers have identified compelling reasons to doubt that the truth is systematic across all domains of thought, arguing that in normative domains, in particular, the truth is largely asystematic. I argue that insofar as the truth in normative domains is asystematic, this renders it correspondingly harder for LLMs to make progress, because they cannot then leverage the systematicity of truth. And the less LLMs can rely on the systematicity of truth, the less we can rely on them to do our practical deliberation for us, because the very asystematicity of normative domains requires human agency to play a greater role in practical thought.