An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problems

TL;DR

This paper discusses how Large Language Models can be used to generate solutions for open-ended problems, going beyond traditional static methods by supporting problem framing, approach exploration, and handling unexpected situations.

Contribution

It provides an overview of current techniques like prompting, reinforcement learning, and retrieval-augmented generation for using LLMs in open-ended problem solving.

Findings

LLMs enable automated solution generation for complex problems.

Current methods include prompting, reinforcement learning, and retrieval-augmented generation.

Future research directions involve improving these techniques for better problem-solving capabilities.

Abstract

Large Language Models offer new opportunities to devise automated implementation generation methods that can tackle problem solving activities beyond traditional methods, which require algorithmic specifications and can use only static domain knowledge, like performance metrics and libraries of basic building blocks. Large Language Models could support creating new methods to support problem solving activities for open-ended problems, like problem framing, exploring possible solving approaches, feature elaboration and combination, more advanced implementation assessment, and handling unexpected situations. This report summarized the current work on Large Language Models, including model prompting, Reinforcement Learning, and Retrieval-Augmented Generation. Future research requirements were also discussed.