Particle swarm optimization with Applications to Maximum Likelihood Estimation and Penalized Negative Binomial Regression

TL;DR

This paper introduces Particle Swarm Optimization (PSO) as a versatile alternative for maximum likelihood estimation and penalized regression, demonstrating its advantages over traditional routines in various complex statistical models.

Contribution

The paper presents PSO as a novel, flexible optimization method capable of handling nonstandard distributions and convergence issues in statistical estimation.

Findings

PSO can reproduce or improve upon traditional estimation results.

PSO successfully estimates parameters where standard routines fail.

PSO offers flexibility in modeling and provides superior estimates in complex models.

Abstract

General purpose optimization routines such as nlminb, optim (R) or nlmixed (SAS) are frequently used to estimate model parameters in nonstandard distributions. This paper presents Particle Swarm Optimization (PSO), as an alternative to many of the current algorithms used in statistics. We find that PSO can not only reproduce the same results as the above routines, it can also produce results that are more optimal or when others cannot converge. In the latter case, it can also identify the source of the problem or problems. We highlight advantages of using PSO using four examples, where: (1) some parameters in a generalized distribution are unidentified using PSO when it is not apparent or computationally manifested using routines in R or SAS; (2) PSO can produce estimation results for the log-binomial regressions when current routines may not; (3) PSO provides flexibility in the link function for binomial regression with LASSO penalty, which is unsupported by standard packages like GLM and GENMOD in Stata and SAS, respectively, and (4) PSO provides superior MLE estimates for an EE-IW distribution compared with those from the traditional statistical methods that rely on moments.