The application of Monte Carlo methods for learning generalized linear model

TL;DR

This paper explores the use of Monte Carlo methods, specifically Metropolis Hastings and Stochastic Approximation, for estimating coefficients in Generalized Linear Models like Logistic Regression, offering an alternative to traditional MLE methods.

Contribution

It introduces Monte Carlo techniques for GLM coefficient estimation, providing a novel approach compared to standard maximum likelihood estimation.

Findings

Monte Carlo methods effectively estimate GLM coefficients.

Metropolis Hastings and SAMC outperform traditional methods in certain scenarios.

Results demonstrate comparable accuracy to MLE with potential computational advantages.

Abstract

Monte Carlo method is a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results. They are often used in physical and mathematical problems and are most useful when it is difficult or impossible to use other mathematical methods. Basically, many statisticians have been increasingly drawn to Monte Carlo method in three distinct problem classes: optimization, numerical integration, and generating draws from a probability distribution. In this paper, we will introduce the Monte Carlo method for calculating coefficients in Generalized Linear Model(GLM), especially for Logistic Regression. Our main methods are Metropolis Hastings(MH) Algorithms and Stochastic Approximation in Monte Carlo Computation(SAMC). For comparison, we also get results automatically using MLE method in R software.