WiCER: Wiki-memory Compile, Evaluate, Refine Iterative Knowledge Compilation for LLM Wiki Systems
Juan M. Huerta

TL;DR
This paper introduces WiCER, an iterative method to improve knowledge compilation for LLM Wiki systems, addressing the compilation gap and enhancing factual retention and retrieval performance.
Contribution
WiCER employs a CEGAR-inspired iterative process to evaluate and refine compiled wikis, significantly reducing factual loss and catastrophic failures in LLM knowledge systems.
Findings
WiCER recovers 80% of lost quality in compiled wikis.
Targeted diagnosis improves compilation quality more than generic pinning.
WiCER reduces catastrophic failures by 55% across multiple domains.
Abstract
The LLM Wiki pattern, to compile and provide domain knowledge into a persistent artifact and serve it to LLMs via KV cache inference, promises context access at sub-second latency with zero retrieval failure. Realizing this requires solving the compilation gap: LLM compilation distilling raw documents into a wiki without catastrophically discarding critical facts. We characterize this gap across 17 RepLiQA domains (6,800 questions): we observe that full context KV cache inference outperforms RAG on curated knowledge (4.38 vs. 4.08 out of 5, 7.3 faster TTFT) but degrades below RAG at scale due to attention dilution, and blind compilation fails entirely (2.14 to 2.32 vs. 3.46, 53 to 60% catastrophic failure rate). To address the compilation gap, we propose WiCER (Wiki-memory Compile, Evaluate, Refine), an iterative algorithm inspired by counterexample-guided abstraction refinement (CEGAR)…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
