TextualVerifier: Verify TextGrad Step-by-Step
Eugenius Mario Situmorang, Adila Alfa Krisnadhi, Ari Wibisono

TL;DR
TextualVerifier enhances TextGrad by providing a self-verification framework using large language models, significantly improving reasoning validity and accuracy in text-based AI optimization tasks without numerical gradients.
Contribution
It introduces the first self-verification framework for TextGrad that leverages chain-of-thought reasoning and majority voting with LLMs, improving reasoning validity and optimization reliability.
Findings
29% increase in reasoning step validity
2.2 percentage point improvement in TextGrad loss integration
Significant performance gains across multiple benchmarks
Abstract
TextGrad is a novel approach to text-based automatic differentiation that enables composite AI systems to perform optimization without explicit numerical equations. However, it currently lacks self-verification mechanisms that ensure reasoning validity in text-based decision making. This research introduces TextualVerifier, a verification framework that leverages chain-of-thought reasoning and majority voting with large language models to address this verification gap. TextualVerifier implements a four-stage workflow: chain-of-thought decomposition, variant generation, majority voting, and consensus aggregation. It integrates non-invasively with TextGrad at both the loss function and optimization result verification stages. Experimental evaluation using the Gemini 1.5 Pro model is conducted in two phases: (1) standalone evaluation on PRM800K, and (2) integrated evaluation with TextGrad…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Machine Learning and Data Classification · Advanced Multi-Objective Optimization Algorithms
