Towards Sensitivity-Aware Language Models

TL;DR

This paper formalizes sensitivity awareness in large language models, connects it to differential privacy, and introduces a fine-tuning method that enhances sensitivity awareness while maintaining overall task performance.

Contribution

It provides a theoretical link between sensitivity awareness and differential privacy and proposes a supervised fine-tuning approach to improve sensitivity awareness in quantized LLMs.

Findings

Up to 21.7% performance improvement in sensitivity awareness

Outperforms similar-sized models in sensitivity awareness tasks

Largely preserves general task performance after fine-tuning

Abstract

With LLMs increasingly deployed in corporate data management, it is crucial to ensure that these models do not leak sensitive information. In the context of corporate data management, the concept of sensitivity awareness has been introduced, enabling LLMs to adhere to predefined access rights rules. However, it remains unclear how sensitivity awareness relates to established notions of privacy, such as differential privacy (DP), thereby making it difficult to deploy meaningfully in real-world applications. In this work, we formalize the notion of sensitivity awareness and theoretically establish its connection to DP. Additionally, we develop a supervised fine-tuning recipe to make existing, four-bit quantized LLMs more sensitivity-aware. With a performance boost of up to 21.7%, the finetuned LLMs not only substantially improve over their baseline but also outperform other full-precision open-source and commercial models of similar size in achieving sensitivity awareness, demonstrating the effectiveness of our proposed approach. At the same time, our method also largely preserves the models' performance on other tasks, such as general instruction-following, mathematical, and common-sense reasoning.