Stingy Context: 18:1 Hierarchical Code Compression for LLM Auto-Coding

TL;DR

Stingy Context is a hierarchical code compression method that significantly reduces LLM context size, enabling efficient auto-coding with high success rates across multiple models and real-world issues.

Contribution

The paper introduces a novel hierarchical compression scheme called Stingy Context, achieving 18:1 reduction in LLM context size for auto-coding tasks.

Findings

Achieves 18:1 compression ratio reducing 239k tokens to 11k tokens

94-97% success rate on 40 real-world issues across 12 models

Outperforms flat compression methods and reduces lost-in-the-middle effects

Abstract

We introduce Stingy Context, a hierarchical tree-based compression scheme achieving 18:1 reduction in LLM context for auto-coding tasks. Using our TREEFRAG exploit decomposition, we reduce a real source code base of 239k tokens to 11k tokens while preserving task fidelity. Empirical results across 12 Frontier models show 94 to 97% success on 40 real-world issues at low cost, outperforming flat methods and mitigating lost-in-the-middle effects.