UPAR: A Kantian-Inspired Prompting Framework for Enhancing Large Language Model Capabilities

TL;DR

The paper introduces UPAR, a Kantian-inspired prompting framework for large language models that enhances interpretability and accuracy by mimicking human cognition through structured phases, leading to significant performance improvements.

Contribution

It presents a novel four-phase prompting framework based on Kantian philosophy, providing a systematic epistemological foundation and improving LLM inference quality.

Findings

Increases GSM8K accuracy from 22.92% to 58.33%.

Improves causal judgment accuracy from 67.91% to 75.40%.

Outperforms existing prompting methods on SCIBENCH without few-shot examples.

Abstract

Large Language Models (LLMs) have demonstrated impressive inferential capabilities, with numerous research endeavors devoted to enhancing this capacity through prompting. Despite these efforts, a unified epistemological foundation is still conspicuously absent. Drawing inspiration from Kant's a priori philosophy, we propose the UPAR prompting framework, designed to emulate the structure of human cognition within LLMs. The UPAR framework is delineated into four phases: "Understand", "Plan", "Act", and "Reflect", enabling the extraction of structured information from complex contexts, prior planning of solutions, execution according to plan, and self-reflection. This structure significantly augments the explainability and accuracy of LLM inference, producing a human-understandable and inspectable inferential trajectory. Furthermore, our work offers an epistemological foundation for existing prompting techniques, allowing for a possible systematic integration of these methods. With GPT-4, our approach elevates the accuracy from COT baseline of 22.92% to 58.33% in a challenging subset of GSM8K, and from 67.91% to 75.40% in the causal judgment task. Without using few-shot examples or external tools, UPAR significantly outperforms existing prompting methods on SCIBENCH, a challenging dataset containing collegiate-level mathematics, chemistry, and physics scientific problems.