Actor Critic with Differentially Private Critic

TL;DR

This paper introduces a differentially private actor-critic reinforcement learning method that enhances sample efficiency in resource-limited tasks while protecting sensitive trajectory data, enabling privacy-preserving knowledge transfer.

Contribution

It presents a novel differentially private critic for actor-critic algorithms, facilitating privacy-preserving transfer learning in reinforcement learning.

Findings

Improved sample efficiency in control tasks

Maintains privacy of trajectory data

Effective knowledge transfer across tasks

Abstract

Reinforcement learning algorithms are known to be sample inefficient, and often performance on one task can be substantially improved by leveraging information (e.g., via pre-training) on other related tasks. In this work, we propose a technique to achieve such knowledge transfer in cases where agent trajectories contain sensitive or private information, such as in the healthcare domain. Our approach leverages a differentially private policy evaluation algorithm to initialize an actor-critic model and improve the effectiveness of learning in downstream tasks. We empirically show this technique increases sample efficiency in resource-constrained control problems while preserving the privacy of trajectories collected in an upstream task.