Navigating the Conceptual Multiverse

TL;DR

This paper introduces the conceptual multiverse, an interactive system for transparently exploring and verifying the decision space in language model outputs, improving understanding and reasoning across domains.

Contribution

It presents a novel framework combining multiverse analysis with verification to make language model decision processes transparent and checkable.

Findings

Participants developed clearer problem maps in three domains.

Philosophy students improved essay framing and thesis reversal.

Poets identified compositional patterns to clarify taste.

Abstract

When language models answer open-ended problems, they implicitly make hidden decisions that shape their outputs, leaving users with uncontextualized answers rather than a working map of the problem; drawing on multiverse analysis from statistics, we build and evaluate the conceptual multiverse, an interactive system that represents conceptual decisions such as how to frame a question or what to value as a space users can transparently inspect, intervenably change, and check against principled domain reasoning; for this structure to be worth navigating rather than misleading, it must be rigorous and checkable against domain reasoning norms, so we develop a general verification framework that enforces properties of good decision structures like unambiguity and completeness calibrated by expert-level reasoning; across three domains, the conceptual multiverse helped participants develop a working map of the problem, with philosophy students rewriting essays with sharper framings and reversed theses, alignment annotators moving from surface preferences to reasoning about user intent and harm, and poets identifying compositional patterns that clarified their taste.