Algebraic Geometrical Analysis of Metropolis Algorithm When Parameters Are Non-identifiable

TL;DR

This paper analyzes the Metropolis algorithm's behavior in non-identifiable models using algebraic geometry, deriving an analytical expression for the average acceptance rate to optimize step size.

Contribution

It provides a novel algebraic geometric framework to understand and optimize the Metropolis algorithm in non-identifiable statistical models.

Findings

Derived an analytical formula for average acceptance rate in non-identifiable cases

Validated the theoretical results through numerical experiments

Proposed an optimization principle for step size based on acceptance rate

Abstract

The Metropolis algorithm is one of the Markov chain Monte Carlo (MCMC) methods that realize sampling from the target probability distribution. In this paper, we are concerned with the sampling from the distribution in non-identifiable cases that involve models with Fisher information matrices that may fail to be invertible. The theoretical adjustment of the step size, which is the variance of the candidate distribution, is difficult for non-identifiable cases. In this study, to establish such a principle, the average acceptance rate, which is used as a guideline to optimize the step size in the MCMC method, was analytically derived in non-identifiable cases. The optimization principle for the step size was developed from the viewpoint of the average acceptance rate. In addition, we performed numerical experiments on some specific target distributions to verify the effectiveness of our theoretical results.