Privacy enabled Financial Text Classification using Differential Privacy and Federated Learning

TL;DR

This paper presents a privacy-preserving approach for financial text classification using BERT and RoBERTa models integrated with Differential Privacy and Federated Learning, enabling secure handling of sensitive financial data.

Contribution

It introduces a novel combination of transformer-based NLP models with privacy techniques specifically tailored for financial text classification tasks.

Findings

Effective privacy-utility tradeoffs demonstrated

Models successfully trained with privacy guarantees

Evaluation on Financial Phrase Bank dataset shows promising results

Abstract

Privacy is important considering the financial Domain as such data is highly confidential and sensitive. Natural Language Processing (NLP) techniques can be applied for text classification and entity detection purposes in financial domains such as customer feedback sentiment analysis, invoice entity detection, categorisation of financial documents by type etc. Due to the sensitive nature of such data, privacy measures need to be taken for handling and training large models with such data. In this work, we propose a contextualized transformer (BERT and RoBERTa) based text classification model integrated with privacy features such as Differential Privacy (DP) and Federated Learning (FL). We present how to privately train NLP models and desirable privacy-utility tradeoffs and evaluate them on the Financial Phrase Bank dataset.