A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworks

TL;DR

This survey comprehensively reviews LLM test-time compute via search, unifying task definitions, profiling, and search algorithms to enable better comparison and understanding of current frameworks and their performance.

Contribution

It introduces a unified framework for LLM inference via search, standardizing task, profiling, and search procedure definitions for clearer comparison.

Findings

Provides a unified MDP-based task definition

Highlights differences in search algorithms from standard methods

Discusses applicability and efficiency of various frameworks

Abstract

LLM test-time compute (or LLM inference) via search has emerged as a promising research area with rapid developments. However, current frameworks often adopt distinct perspectives on three key aspects: task definition, LLM profiling, and search procedures, making direct comparisons challenging. Moreover, the search algorithms employed often diverge from standard implementations, and their specific characteristics are not thoroughly specified. This survey aims to provide a comprehensive but integrated technical review on existing LIS frameworks. Specifically, we unify task definitions under Markov Decision Process (MDP) and provides modular definitions of LLM profiling and search procedures. The definitions enable precise comparisons of various LLM inference frameworks while highlighting their departures from conventional search algorithms. We also discuss the applicability, performance, and efficiency of these methods. For ongoing paper updates, please refer to our GitHub repository: https://github.com/xinzhel/LLM-Search.