Incremental Mixture Importance Sampling with Shotgun optimization

TL;DR

This paper introduces Shotgun optimization, a framework combining multiple optimization strategies within an incremental mixture importance sampling algorithm to improve posterior sampling in complex, multimodal, and challenging likelihood scenarios.

Contribution

It develops a novel Shotgun optimization framework integrated into importance sampling, enhancing exploration and robustness in multimodal and complex models.

Findings

Improved posterior samples for multimodal densities.

Enhanced robustness when likelihood and prior disagree.

Effective in parameter estimation from differential equation models.

Abstract

This paper proposes a general optimization strategy, which combines results from different optimization or parameter estimation methods to overcome shortcomings of a single method. Shotgun optimization is developed as a framework which employs different optimization strategies, criteria, or conditional targets to enable wider likelihood exploration. The introduced Shotgun optimization approach is embedded into an incremental mixture importance sampling algorithm to produce improved posterior samples for multimodal densities and creates robustness in cases where the likelihood and prior are in disagreement. Despite using different optimization approaches, the samples are combined into samples from a single target posterior. The diversity of the framework is demonstrated on parameter estimation from differential equation models employing diverse strategies including numerical solutions and approximations thereof. Additionally the approach is demonstrated on mixtures of discrete and continuous parameters and is shown to ease estimation from synthetic likelihood models. R code of the implemented examples is stored in a zipped archive (codeSubmit.zip).