Direct Fisher Score Estimation for Likelihood Maximization

TL;DR

This paper introduces a novel gradient-based optimization method that directly estimates the Fisher score using local score matching and simulations, enabling efficient likelihood maximization in complex models.

Contribution

It presents a new sequential, simulation-based approach for likelihood maximization that directly models the Fisher score with theoretical guarantees and practical efficiency.

Findings

Outperforms existing methods on synthetic problems

Provides theoretical bounds on score estimator bias

Demonstrates effectiveness on real-world datasets

Abstract

We study the problem of likelihood maximization when the likelihood function is intractable but model simulations are readily available. We propose a sequential, gradient-based optimization method that directly models the Fisher score based on a local score matching technique which uses simulations from a localized region around each parameter iterate. By employing a linear parameterization to the surrogate score model, our technique admits a closed-form, least-squares solution. This approach yields a fast, flexible, and efficient approximation to the Fisher score, effectively smoothing the likelihood objective and mitigating the challenges posed by complex likelihood landscapes. We provide theoretical guarantees for our score estimator, including bounds on the bias introduced by the smoothing. Empirical results on a range of synthetic and real-world problems demonstrate the superior performance of our method compared to existing benchmarks.