PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring

TL;DR

PackMonitor is a novel, decoding-time monitoring approach that guarantees zero package hallucinations in LLMs during dependency recommendations by leveraging authoritative package lists and real-time intervention.

Contribution

This work introduces PackMonitor, the first method to fundamentally eliminate package hallucinations in LLMs through continuous monitoring and intervention during decoding.

Findings

Achieves zero package hallucinations across five LLMs.

Maintains low latency and model capabilities.

Scalable to millions of packages with negligible overhead.

Abstract

As Large Language Models (LLMs) are increasingly integrated into software development workflows, their trustworthiness has become a critical concern. However, in dependency recommendation scenarios, the reliability of LLMs is undermined by widespread package hallucinations, where models often recommend hallucinated packages. Recent studies have proposed a range of approaches to mitigate this issue. Nevertheless, existing approaches typically merely reduce hallucination rates rather than eliminate them, leaving persistent software security risks. In this work, we argue that package hallucinations are theoretically preventable based on the key insight that package validity is decidable through finite and enumerable authoritative package lists. Building on this, we propose PackMonitor, the first approach capable of fundamentally eliminating package hallucinations by continuously monitoring the model's decoding process and intervening when necessary. To implement this in practice, PackMonitor addresses three key challenges: (1) determining when to trigger intervention via a Context-Aware Parser that continuously monitors model outputs and selectively activates intervening only during installation command generation; (2) resolving how to intervene by employing a Package-Name Intervenor that strictly limits the decoding space to an authoritative package list; and (3) ensuring monitoring efficiency through a DFA-Caching Mechanism that enables scalability to millions of packages with negligible overhead. Extensive experiments on five widely used LLMs demonstrate that PackMonitor is a training-free, plug-and-play solution that consistently reduces package hallucination rates to zero while maintaining low-latency inference and preserving original model capabilities.