JoPA:Explaining Large Language Model's Generation via Joint Prompt Attribution

TL;DR

This paper introduces JoPA, a novel framework for explaining how multiple prompts collaboratively influence large language model outputs, addressing the complexity of prompt interactions in text generation.

Contribution

JoPA formulates prompt attribution as a combinatorial optimization problem and proposes a probabilistic algorithm to identify influential prompt combinations for generation explanation.

Findings

JoPA effectively explains prompt influence on generation.

The framework demonstrates high faithfulness in explanations.

JoPA is efficient in identifying key prompt combinations.

Abstract

Large Language Models (LLMs) have demonstrated impressive performances in complex text generation tasks. However, the contribution of the input prompt to the generated content still remains obscure to humans, underscoring the necessity of understanding the causality between input and output pairs. Existing works for providing prompt-specific explanation often confine model output to be classification or next-word prediction. Few initial attempts aiming to explain the entire language generation often treat input prompt texts independently, ignoring their combinatorial effects on the follow-up generation. In this study, we introduce a counterfactual explanation framework based on Joint Prompt Attribution, JoPA, which aims to explain how a few prompt texts collaboratively influences the LLM's complete generation. Particularly, we formulate the task of prompt attribution for generation interpretation as a combinatorial optimization problem, and introduce a probabilistic algorithm to search for the casual input combination in the discrete space. We define and utilize multiple metrics to evaluate the produced explanations, demonstrating both the faithfulness and efficiency of our framework.