TL;DR
NCO is a decoding strategy that efficiently enforces multiple hard and regex constraints during language model generation, reducing computational costs and supporting various inference methods.
Contribution
It introduces NCO, a novel online pattern matching decoding method that handles multiple constraints without state explosion, compatible with standard inference strategies.
Findings
Effective in suppressing PII and profanity in LLM outputs.
Reduces computational overhead compared to traditional constrained decoding.
Supports soft masking for probabilistic content suppression.
Abstract
Controlling Large Language Models (LLMs) to prevent the generation of undesirable content, such as profanity and personally identifiable information (PII), has become increasingly critical. While earlier approaches relied on post-processing or resampling, recent research has shifted towards constrained decoding methods that control outputs during generation to mitigate high computational costs and quality degradation. However, preventing multiple forbidden hard constraints or regex constraints from appearing anywhere in the output is computationally challenging. A straightforward solution is to convert these constraints into a single automaton that tracks all forbidden patterns during decoding, but this often becomes impractically large. Standard regex engines also do not readily support the operations needed to build such a constraint, such as complement and intersection. In order to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
