Unleashing the Creative Mind: Language Model As Hierarchical Policy For Improved Exploration on Challenging Problem Solving
Zhan Ling, Yunhao Fang, Xuanlin Li, Tongzhou Mu, Mingu Lee, Reza, Pourreza, Roland Memisevic, Hao Su

TL;DR
This paper introduces a hierarchical policy framework for large language models that enhances exploration of diverse problem-solving strategies, leading to improved accuracy on challenging reasoning tasks by using a visionary leader and a detailed follower with a tournament-based selection.
Contribution
It proposes framing LLMs as hierarchical policies with high-level hints and detailed execution, significantly improving exploration and solution accuracy on complex problems.
Findings
Enhanced exploration of diverse strategies in LLMs.
Improved accuracy on challenging math problems.
Effective tournament-based solution selection.
Abstract
Large Language Models (LLMs) have achieved tremendous progress, yet they still often struggle with challenging reasoning problems. Current approaches address this challenge by sampling or searching detailed and low-level reasoning chains. However, these methods are still limited in their exploration capabilities, making it challenging for correct solutions to stand out in the huge solution space. In this work, we unleash LLMs' creative potential for exploring multiple diverse problem solving strategies by framing an LLM as a hierarchical policy via in-context learning. This policy comprises of a visionary leader that proposes multiple diverse high-level problem-solving tactics as hints, accompanied by a follower that executes detailed problem-solving processes following each of the high-level instruction. The follower uses each of the leader's directives as a guide and samples multiple…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques
