Large Language Model Programs

TL;DR

This paper explores embedding large language models within algorithms to enhance their capabilities, demonstrating a 6.4% improvement in evidence-supported question-answering without finetuning.

Contribution

It introduces a method to embed LLMs in algorithms, expanding their functionality beyond traditional in-context learning approaches.

Findings

6.4% improvement over chain of thought baseline

Enhanced question-answering performance without finetuning

Discussion of advantages and disadvantages of the approach

Abstract

In recent years, large pre-trained language models (LLMs) have demonstrated the ability to follow instructions and perform novel tasks from a few examples. The possibility to parameterise an LLM through such in-context examples widens their capability at a much lower cost than finetuning. We extend this line of reasoning and present a method which further expands the capabilities of an LLM by embedding it within an algorithm or program. To demonstrate the benefits of this approach, we present an illustrative example of evidence-supported question-answering. We obtain a 6.4\% improvement over the chain of thought baseline through a more algorithmic approach without any finetuning. Furthermore, we highlight recent work from this perspective and discuss the advantages and disadvantages in comparison to the standard approaches.