From Tool Calling to Symbolic Thinking: LLMs in a Persistent Lisp Metaprogramming Loop

TL;DR

This paper introduces a new architecture combining large language models with a persistent Lisp environment, enabling dynamic tool creation, reflective programming, and stateful interactions for more interactive AI systems.

Contribution

It presents a novel framework for integrating LLMs with a live Lisp environment, facilitating self-evolving tool development and symbolic reasoning in AI.

Findings

Enables LLMs to define and invoke tools within a Lisp REPL.

Supports stateful external memory and reflective programming.

Provides architectural principles for future interactive AI systems.

Abstract

We propose a novel architecture for integrating large language models (LLMs) with a persistent, interactive Lisp environment. This setup enables LLMs to define, invoke, and evolve their own tools through programmatic interaction with a live REPL. By embedding Lisp expressions within generation and intercepting them via a middleware layer, the system allows for stateful external memory, reflective programming, and dynamic tool creation. We present a design framework and architectural principles to guide future implementations of interactive AI systems that integrate symbolic programming with neural language generation.