Reducing Hallucinations in LLM-Generated Code via Semantic Triangulation

TL;DR

This paper proposes semantic triangulation, a framework that reduces hallucinations in LLM-generated code by checking consistency between solutions to transformed problem variants, improving correctness confidence.

Contribution

It introduces a theory-grounded framework with four concrete methods that decorrelate model errors and enhance program correctness verification in code generation.

Findings

Increases correct program selection probability by 24% over baselines

Achieves 26% higher F1 score in selection-or-abstention scenarios

Consistently handles inexact problems with multiple valid solutions

Abstract

Large language models (LLMs) can generate executable code from natural language descriptions, but the resulting programs frequently contain bugs due to hallucinations. In the absence of formal specifications, existing approaches attempt to assess correctness using LLM-generated proxies such as tests or auto-formalized specifications. However, these proxies are produced by the same imperfect models and thus often corroborate rather than catch errors, especially when the model exhibits correlated errors. We introduce semantic triangulation, a theory-grounded framework that decorrelates model errors by transforming the original problem into a dissociative variant - one likely requiring a fundamentally different algorithm - and checks consistency between independently sampled solutions to both problems. We identify theoretical requirements for this framework, and we prove that under a formal model of LLM hallucinations, these properties confer higher confidence in program correctness. We instantiate the framework through four concrete triangulation methods based on problem inversion, decomposition, and solution enumeration. Evaluated on LiveCodeBench and CodeElo across GPT-4o, DeepSeek-V3, and Gemini 2.5 Flash, our tool increases the probability of selecting a correct program by 24% over baselines (test generation, metamorphic testing, and auto-formalized specifications) and achieves 26% higher F1 score in selection-or-abstention scenarios, while being the only method that consistently handles inexact problems admitting multiple valid solutions.