Permissive Information-Flow Analysis for Large Language Models

TL;DR

This paper introduces a permissive information-flow analysis method for large language models that selectively propagates influential input labels, enhancing security and privacy without being overly conservative.

Contribution

It proposes a novel, less restrictive label propagation technique for LLMs, improving security analysis by focusing on influential inputs rather than all data.

Findings

Improves information flow analysis in LLMs in over 85% of cases.

Demonstrates effectiveness of prompt-based and k-NN based label propagation.

Outperforms baseline introspection method in experimental evaluations.

Abstract

Large Language Models (LLMs) are rapidly becoming commodity components of larger software systems. This poses natural security and privacy problems: poisoned data retrieved from one component can change the model's behavior and compromise the entire system, including coercing the model to spread confidential data to untrusted components. One promising approach is to tackle this problem at the system level via dynamic information flow (aka taint) tracking. Unfortunately, this approach of propagating the most restrictive input label to the output is too conservative for applications where LLMs operate on inputs retrieved from diverse sources. In this paper, we propose a novel, more permissive approach to propagate information flow labels through LLM queries. The key idea behind our approach is to propagate only the labels of the samples that were influential in generating the model output and to eliminate the labels of unnecessary inputs. We implement and investigate the effectiveness of two variations of this approach, based on (i) prompt-based retrieval augmentation, and (ii) a $k$-nearest-neighbors language model. We compare these with a baseline that uses introspection to predict the output label. Our experimental results in an LLM agent setting show that the permissive label propagator improves over the baseline in more than 85% of the cases, which underscores the practicality of our approach.