Context-Masked Meta-Prompting for Privacy-Preserving LLM Adaptation in Finance

TL;DR

This paper introduces a privacy-preserving meta-prompting technique for adapting large language models to financial tasks, ensuring confidentiality while improving prompt effectiveness using a novel regeneration process.

Contribution

It proposes a new iterative meta-prompting method with feeder and propagation techniques to optimize prompts without revealing sensitive context, enhancing LLM performance in finance.

Findings

Achieved over 103% improvement in ROUGE-L F1 for financial question answering

Demonstrated effectiveness on datasets like SQuAD, CNN/DailyMail, and SAMSum

Provides a low-cost, privacy-preserving approach for LLM adaptation in finance

Abstract

The increasing reliance on Large Language Models (LLMs) in sensitive domains like finance necessitates robust methods for privacy preservation and regulatory compliance. This paper presents an iterative meta-prompting methodology designed to optimise hard prompts without exposing proprietary or confidential context to the LLM. Through a novel regeneration process involving feeder and propagation methods, we demonstrate significant improvements in prompt efficacy. Evaluated on public datasets serving as proxies for financial tasks such as SQuAD for extractive financial Q&A, CNN/DailyMail for news summarisation, and SAMSum for client interaction summarisation, our approach, utilising GPT-3.5 Turbo, achieved a 103.87% improvement in ROUGE-L F1 for question answering. This work highlights a practical, low-cost strategy for adapting LLMs to financial applications while upholding critical privacy and auditability standards, offering a compelling case for its relevance in the evolving landscape of generative AI in finance.