Unlocking Large Language Model's Planning Capabilities with Maximum Diversity Fine-tuning

TL;DR

This paper explores how fine-tuning large language models with diverse, representative data improves their planning abilities, introducing efficient sampling methods that outperform existing approaches across multiple benchmarks.

Contribution

It proposes the CMDS algorithm for selecting diverse fine-tuning data, including a graph-based variant, significantly enhancing planning performance with fewer samples.

Findings

CMDS outperforms random sampling in fine-tuning.

Graph-based CMDS-g consistently improves planning accuracy.

Fine-tuning with diverse data boosts LLMs' planning capabilities.

Abstract

Large language models (LLMs) have demonstrated impressive task-solving capabilities through prompting techniques and system designs, including solving planning tasks (e.g., math proofs, basic travel planning) when sufficient data is available online and used during pre-training. However, for planning tasks with limited prior data (e.g., blocks world, advanced travel planning), the performance of LLMs, including proprietary models like GPT and Gemini, is poor. This paper investigates the impact of fine-tuning on the planning capabilities of LLMs, revealing that LLMs can achieve strong performance in planning through substantial (tens of thousands of specific examples) fine-tuning. Yet, this process incurs high economic, time, and computational costs for each planning problem variation. To address this, we propose Clustering-Based Maximum Diversity Sampling (CMDS), which selects diverse and representative data to enhance sample efficiency and the model's generalization capability. Extensive evaluations demonstrate that CMDS-l, a baseline method combining CMDS with language embeddings, outperforms random sampling. Furthermore, we introduce a novel algorithm, CMDS-g, which encodes planning task instances with their graph representations into the embedding space. Empirical results show that CMDS-g consistently outperforms baseline methods across various scales and multiple benchmark domains.