Code Hallucination

TL;DR

This paper explores the phenomenon of code hallucination in large language models, introduces a technique called HallTrigger to generate such hallucinations efficiently, and discusses their impact on software development.

Contribution

The paper identifies different types of code hallucination, and presents HallTrigger, a novel prompt-based method to reliably induce hallucinations in blackbox LLMs.

Findings

HallTrigger effectively triggers hallucinations in popular LLMs.

Code hallucination significantly impacts software development.

Various types of code hallucination are characterized.

Abstract

Generative models such as large language models are extensively used as code copilots and for whole program generation. However, the programs they generate often have questionable correctness, authenticity and reliability in terms of integration as they might not follow the user requirements, provide incorrect and/or nonsensical outputs, or even contain semantic/syntactic errors - overall known as LLM hallucination. In this work, we present several types of code hallucination. We have generated such hallucinated code manually using large language models. We also present a technique - HallTrigger, in order to demonstrate efficient ways of generating arbitrary code hallucination. Our method leverages 3 different dynamic attributes of LLMs to craft prompts that can successfully trigger hallucinations from models without the need to access model architecture or parameters. Results from popular blackbox models suggest that HallTrigger is indeed effective and the pervasive LLM hallucination have sheer impact on software development.