Introspection of Thought Helps AI Agents

TL;DR

This paper introduces INoT, a novel AI reasoning framework that enables LLMs to perform introspective, programmatic reasoning within the model, reducing inference costs and improving performance across multiple tasks.

Contribution

The paper proposes INoT, a new LLM prompt design that allows self-reflection and programmatic reasoning, enhancing AI agent capabilities while lowering token costs.

Findings

Average performance improvement of 7.95% over baselines

Token cost reduced by 58.3% compared to top baseline methods

Effective in text and image interpretation tasks

Abstract

AI Agents rely on Large Language Models (LLMs) and Multimodal-LLMs (MLLMs) to perform interpretation and inference in text and image tasks without post-training, where LLMs and MLLMs play the most critical role and determine the initial ability and limitations of AI Agents. Usually, AI Agents utilize sophisticated prompt engineering and external reasoning framework to obtain a promising interaction with LLMs, e.g., Chain-of-Thought, Iteration of Thought and Image-of-Thought. However, they are still constrained by the inherent limitations of LLM in understanding natural language, and the iterative reasoning process will generate a large amount of inference cost. To this end, we propose a novel AI Agent Reasoning Framework with Introspection of Thought (INoT) by designing a new LLM-Read code in prompt. It enables LLM to execute programmatic dialogue reasoning processes following the code in prompt. Therefore, self-denial and reflection occur within LLM instead of outside LLM, which can reduce token cost effectively. Through our experiments on six benchmarks for three different tasks, the effectiveness of INoT is verified, with an average improvement of 7.95\% in performance, exceeding the baselines. Furthermore, the token cost of INoT is lower on average than the best performing method at baseline by 58.3\%. In addition, we demonstrate the versatility of INoT in image interpretation and inference through verification experiments.