On fundamental properties of high-order forward-backward envelope

TL;DR

This paper explores the fundamental mathematical properties of high-order forward-backward splitting mappings and their envelopes, providing theoretical insights crucial for designing gradient-based algorithms for nonconvex composite optimization.

Contribution

It establishes key properties of HiFBS and HiFBE, including boundedness, continuity, subdifferential forms, and conditions for differentiability, advancing the theoretical foundation for optimization algorithms.

Findings

Proved boundedness and continuity of HiFBS and HiFBE.

Derived explicit subdifferential forms for HiFBE.

Identified conditions for differentiability and smoothness of HiFBE.

Abstract

This paper studies the fundamental properties of the high-order forward-backward splitting mapping (HiFBS) and its associated forward-backward envelope (HiFBE) through the lens of high-order regularization for nonconvex composite functions. Specifically, we (i) establish the boundedness and uniform boundedness of HiFBS, along with the H\"older and Lipschitz continuity of HiFBE; (ii) derive an explicit form for the subdifferentials of HiFBE; and (iii) investigate necessary and sufficient conditions for the differentiability and weak smoothness of HiFBE under suitable assumptions. By leveraging the prox-regularity of $g$ and the concept of $p$-calmness, we further demonstrate the local single-valuedness and continuity of HiFBS, which in turn guarantee the differentiability of HiFBE in neighborhoods of calm points. This paves the way for the development of gradient-based algorithms tailored to nonconvex composite optimization problems.