Global Optimization By Gradient From Hierarchical Score-Matching Spaces

TL;DR

This paper introduces a unified hierarchical optimization framework that uses score-matching gradients to address complex constraints and local optima issues, linking global optimization with diffusion-based generative models.

Contribution

It proposes a novel hierarchical optimization approach that overcomes traditional limitations of gradient methods by unifying constrained problems and leveraging score-matching gradients.

Findings

Successfully applied to simple and complex experiments

Reveals a connection between global optimization and diffusion models

Outperforms traditional gradient methods in complex scenarios

Abstract

Gradient-based methods are widely used to solve various optimization problems, however, they are either constrained by local optima dilemmas, simple convex constraints, and continuous differentiability requirements, or limited to low-dimensional simple problems. This work solve these limitations and restrictions by unifying all optimization problems with various complex constraints as a general hierarchical optimization objective without constraints, which is optimized by gradient obtained through score matching. The proposed method is verified through simple-constructed and complex-practical experiments. Even more importantly, it reveals the profound connection between global optimization and diffusion based generative modeling.