# Law of the Iterated Logarithm and Model Selection Consistency for GLMs with Independent and Dependent Responses

## TL;DR

This paper establishes the law of the iterated logarithm for maximum likelihood estimators in generalized linear models with both independent and dependent responses, and applies it to prove the strong consistency of model selection criteria like BIC.

## Contribution

It extends the law of the iterated logarithm to GLMs with dependent responses and demonstrates the resulting strong consistency of penalized likelihood model selection methods.

## Key findings

- LIL holds for MLE in GLMs with dependent responses.
- Penalized likelihood criteria can consistently select the correct model.
- Simulation confirms BIC's selection consistency.

## Abstract

We study the law of the iterated logarithm (LIL) for the maximum likelihood estimation of the parameters (as a convex optimization problem) in the generalized linear models with independent or weakly dependent ($\rho$-mixing, $m$-dependent) responses under mild conditions. The LIL is useful to derive the asymptotic bounds for the discrepancy between the empirical process of the log-likelihood function and the true log-likelihood. As the application of the LIL, the strong consistency of some penalized likelihood based model selection criteria can be shown. Under some regularity conditions, the model selection criterion will be helpful to select the simplest correct model almost surely when the penalty term increases with model dimension and the penalty term has an order higher than $O({\rm{loglog}}n)$ but lower than $O(n)$. Simulation studies are implemented to verify the selection consistency of BIC.