Language Model Cascades

TL;DR

This paper formalizes the concept of language model cascades, which are composed probabilistic programs that enhance few-shot learning by combining multiple models and techniques like control flow and inference strategies.

Contribution

It introduces a unified formal framework for various existing techniques such as scratchpads, verifiers, and tool use, viewing them as language model cascades.

Findings

Formalization of language model techniques as probabilistic programs

Unified perspective on diverse methods like chain of thought and tool use

Framework enables systematic design and analysis of model compositions

Abstract

Prompted models have demonstrated impressive few-shot learning abilities. Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. These compositions are probabilistic models, and may be expressed in the language of graphical models with random variables whose values are complex data types such as strings. Cases with control flow and dynamic structure require techniques from probabilistic programming, which allow implementing disparate model structures and inference strategies in a unified language. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use. We refer to the resulting programs as language model cascades.