Solving Stochastic Optimization by Newton-type methods with Dimension-Adaptive Sparse Grid Quadrature

TL;DR

This paper introduces a dimension-adaptive sparse grid quadrature method to efficiently solve high-dimensional stochastic optimization problems by accurately approximating integrals involved in objective, gradient, and Hessian calculations.

Contribution

It presents a novel application of dimension-adaptive sparse grid quadrature within the 'optimise then discretise' framework for stochastic optimization, demonstrating improved performance over traditional methods.

Findings

Dimension-adaptive sparse grid quadrature achieves high accuracy in high dimensions.

The method outperforms classical sparse grid approaches in stochastic optimization.

It enhances the efficiency of calculating integrals for smooth integrands.

Abstract

Stochastic optimisation problems minimise expectations of random cost functions. We use 'optimise then discretise' method to solve stochastic optimisation. In our approach, accurate quadrature methods are required to calculate the objective, gradient or Hessian which are in fact integrals. We apply the dimension-adaptive sparse grid quadrature to approximate these integrals when the problem is high dimensional. Dimension-adaptive sparse grid quadrature shows high accuracy and efficiency in computing an integral with a smooth integrand. It is a kind of generalisation of the classical sparse grid method, which refines different dimensions according to their importance. We show that the dimension-adaptive sparse grid quadrature has better performance in the optimise then discretise' method than the 'discretise then optimise' method.