Recursive Decomposition with Dependencies for Generic Divide-and-Conquer Reasoning

TL;DR

This paper introduces Recursive Decomposition with Dependencies (RDD), a scalable divide-and-conquer approach for reasoning tasks that requires less supervision, handles sub-task dependencies, and improves performance and efficiency on complex benchmarks.

Contribution

RDD is a novel recursive divide-and-conquer method that manages sub-task dependencies and error recovery, reducing supervision and enhancing scalability for reasoning tasks.

Findings

RDD outperforms existing methods on complex benchmarks.

RDD is more computationally efficient than prior approaches.

RDD works effectively without task-specific guidance.

Abstract

Reasoning tasks are crucial in many domains, especially in science and engineering. Although large language models (LLMs) have made progress in reasoning tasks using techniques such as chain-of-thought and least-to-most prompting, these approaches still do not effectively scale to complex problems in either their performance or execution time. Moreover, they often require additional supervision for each new task, such as in-context examples. In this work, we introduce Recursive Decomposition with Dependencies (RDD), a scalable divide-and-conquer method for solving reasoning problems that requires less supervision than prior approaches. Our method can be directly applied to a new problem class even in the absence of any task-specific guidance. Furthermore, RDD supports sub-task dependencies, allowing for ordered execution of sub-tasks, as well as an error recovery mechanism that can correct mistakes made in previous steps. We evaluate our approach on two benchmarks with six difficulty levels each and in two in-context settings: one with task-specific examples and one without. Our results demonstrate that RDD outperforms other methods in a compute-matched setting as task complexity increases, while also being more computationally efficient.