Deep Loss Convexification for Learning Iterative Models

TL;DR

This paper introduces Deep Loss Convexification (DLC), a method that shapes the loss landscape into a convex-like form around ground truth to improve the training and accuracy of iterative models in tasks like point cloud registration.

Contribution

The paper proposes a novel adversarial training approach using star-convexity constraints to reshape loss landscapes, enabling near-optimal predictions and state-of-the-art results.

Findings

Achieves state-of-the-art performance in 3D point cloud registration.

Effectively reshapes loss landscapes into convex-like forms.

Improves iterative model training stability and accuracy.

Abstract

Iterative methods such as iterative closest point (ICP) for point cloud registration often suffer from bad local optimality (e.g. saddle points), due to the nature of nonconvex optimization. To address this fundamental challenge, in this paper we propose learning to form the loss landscape of a deep iterative method w.r.t. predictions at test time into a convex-like shape locally around each ground truth given data, namely Deep Loss Convexification (DLC), thanks to the overparametrization in neural networks. To this end, we formulate our learning objective based on adversarial training by manipulating the ground-truth predictions, rather than input data. In particular, we propose using star-convexity, a family of structured nonconvex functions that are unimodal on all lines that pass through a global minimizer, as our geometric constraint for reshaping loss landscapes, leading to (1) extra novel hinge losses appended to the original loss and (2) near-optimal predictions. We demonstrate the state-of-the-art performance using DLC with existing network architectures for the tasks of training recurrent neural networks (RNNs), 3D point cloud registration, and multimodel image alignment.