Towards the Holographic Characteristic of LLMs for Efficient Short-text Generation

TL;DR

This paper investigates the Holographic Characteristic of LLMs, revealing their tendency to capture keywords early in generation, and introduces HOLO, a plugin that improves inference efficiency in short-text generation.

Contribution

The paper identifies the Holographic Characteristic of LLMs and proposes HOLO, a plugin that leverages this trait for more efficient short-text generation.

Findings

HOLO achieves comparable performance to baselines.

Language models tend to capture target keywords early.

The Holographic Characteristic is effective across various LLM architectures.

Abstract

The recent advancements in Large Language Models (LLMs) have attracted interest in exploring their in-context learning abilities and chain-of-thought capabilities. However, there are few studies investigating the specific traits related to the powerful generation capacity of LLMs. This paper aims to delve into the generation characteristics exhibited by LLMs. Through our investigation, we have discovered that language models tend to capture target-side keywords at the beginning of the generation process. We name this phenomenon the Holographic Characteristic of language models. For the purpose of exploring this characteristic and further improving the inference efficiency of language models, we propose a plugin called HOLO, which leverages the Holographic Characteristic to extract target-side keywords from language models within a limited number of generation steps and complements the sentence with a parallel lexically constrained text generation method. To verify the effectiveness of HOLO, we conduct massive experiments on language models of varying architectures and scales in the short-text generation scenario. The results demonstrate that HOLO achieves comparable performance to the baselines in terms of both automatic and human-like evaluation metrics and highlight the potential of the Holographic Characteristic.