2nd-order Updates with 1st-order Complexity

TL;DR

This paper introduces VA-Flow, an efficient algorithm that computes second order information at linear complexity, improving inverse kinematics and gradient descent applications by leveraging basic physics and numerical approximations.

Contribution

It presents VA-Flow, a novel method for obtaining second order information with ${\

Findings

VA-Flow is fast and robust for inverse kinematics.

It produces smooth solutions near singularities.

It performs well in gradient descent when the cost function is polynomial-like.

Abstract

It has long been a goal to efficiently compute and use second order information on a function ($f$) to assist in numerical approximations. Here it is shown how, using only basic physics and a numerical approximation, such information can be accurately obtained at a cost of ${\cal O}(N)$ complexity, where $N$ is the dimensionality of the parameter space of $f$. In this paper, an algorithm ({\em VA-Flow}) is developed to exploit this second order information, and pseudocode is presented. It is applied to two classes of problems, that of inverse kinematics (IK) and gradient descent (GD). In the IK application, the algorithm is fast and robust, and is shown to lead to smooth behavior even near singularities. For GD the algorithm also works very well, provided the cost function is locally well-described by a polynomial.