Privacy-Preserving Black-Box Optimization (PBBO): Theory and the Model-Based Algorithm DFOp

TL;DR

This paper introduces DFOp, a novel derivative-free optimization algorithm designed for privacy-preserving black-box problems with encrypted objectives, providing convergence guarantees and improved performance over existing methods.

Contribution

The paper presents DFOp, the first derivative-free solver capable of handling encrypted black-box functions with privacy guarantees, along with new model updating formulas and privacy mechanisms.

Findings

DFOp outperforms existing algorithms in numerical tests.

It guarantees convergence for encrypted objective functions.

Introduces two privacy-preserving noise mechanisms.

Abstract

This paper focuses on solving unconstrained privacy-preserving black-box optimization (PBBO), its corresponding least Frobenius norm updating of quadratic models, and the differentially privacy mechanisms for PBBO. Optimization problems with transformed/encrypted objective functions aim to minimize F(x), which is encrypted/transformed/encrypted to F_k(x) as the output at the k-th iteration. A new derivative-free solver named DFOp, with its implementation, is proposed in this paper, which has a new updating formula for the quadratic model functions. The convergence of DFOp for solving problems with transformed/encrypted objective functions is given. Other analyses, including the new model updating formula and the analysis of the transformation's impact to model functions are presented. We propose two differentially private noise-adding mechanisms for privacy-preserving black-box optimization. Numerical results show that DFOp performs better than compared algorithms. To the best of our knowledge, DFOp is the first derivative-free solver that can solve black-box optimization problems with step-encryption and privacy-preserving black-box problems exactly, which also tries to answer the open question about the combination of derivative-free optimization and privacy.