PERSOMA: PERsonalized SOft ProMpt Adapter Architecture for Personalized Language Prompting

TL;DR

PERSOMA introduces a novel personalized soft prompt adapter architecture that efficiently captures and compresses user interaction history into expressive embeddings, enhancing large language models' ability to adapt to individual user preferences.

Contribution

The paper presents PERSOMA, a new method for personalized prompting that outperforms existing techniques in handling complex user histories using a novel soft prompt adaptation approach.

Findings

PERSOMA outperforms existing embedding and prompt-based personalization methods.

It effectively captures large and complex user interaction histories.

The approach demonstrates superior adaptability in personalized language modeling.

Abstract

Understanding the nuances of a user's extensive interaction history is key to building accurate and personalized natural language systems that can adapt to evolving user preferences. To address this, we introduce PERSOMA, Personalized Soft Prompt Adapter architecture. Unlike previous personalized prompting methods for large language models, PERSOMA offers a novel approach to efficiently capture user history. It achieves this by resampling and compressing interactions as free form text into expressive soft prompt embeddings, building upon recent research utilizing embedding representations as input for LLMs. We rigorously validate our approach by evaluating various adapter architectures, first-stage sampling strategies, parameter-efficient tuning techniques like LoRA, and other personalization methods. Our results demonstrate PERSOMA's superior ability to handle large and complex user histories compared to existing embedding-based and text-prompt-based techniques.