Inappropriate use of L-BFGS, Illustrated on frame field design

TL;DR

This paper critically examines the misuse of L-BFGS in non-convex problems like 2D frame field design, revealing unpredictable performance and highlighting the importance of problem structure in optimization outcomes.

Contribution

It provides an analysis of L-BFGS behavior on complex non-linear problems with multiple local minima, illustrating its limitations and unexpected capabilities in graphics applications.

Findings

L-BFGS can sometimes escape local minima due to Hessian approximation smoothing.

Performance varies significantly depending on sampling on primal vs. dual graphs.

L-BFGS shows unpredictable results in non-convex, highly non-linear problems.

Abstract

L-BFGS is a hill climbing method that is guarantied to converge only for convex problems. In computer graphics, it is often used as a black box solver for a more general class of non linear problems, including problems having many local minima. Some works obtain very nice results by solving such difficult problems with L-BFGS. Surprisingly, the method is able to escape local minima: our interpretation is that the approximation of the Hessian is smoother than the real Hessian, making it possible to evade the local minima. We analyse the behavior of L-BFGS on the design of 2D frame fields. It involves an energy function that is infinitly continuous, strongly non linear and having many local minima. Moreover, the local minima have a clear visual interpretation: they corresponds to differents frame field topologies. We observe that the performances of LBFGS are almost unpredictables: they are very competitive when the field is sampled on the primal graph, but really poor when they are sampled on the dual graph.