Marginal likelihood for parallel series

TL;DR

This paper investigates how the Fisher information about an autocorrelation parameter accumulates when observing multiple parallel series, revealing an unusual decrease in information beyond a certain number of series.

Contribution

It analyzes the behavior of the profile likelihood and Fisher information in Gaussian models with multiple series, highlighting an anomaly in information accumulation.

Findings

Fisher information peaks at about half the series length

Profile likelihood exhibits anomalous behavior with increasing series

Information decreases when the number of series exceeds half the length

Abstract

Suppose that $k$ series, all having the same autocorrelation function, are observed in parallel at $n$ points in time or space. From a single series of moderate length, the autocorrelation parameter $\beta$ can be estimated with limited accuracy, so we aim to increase the information by formulating a suitable model for the joint distribution of all series. Three Gaussian models of increasing complexity are considered, two of which assume that the series are independent. This paper studies the rate at which the information for $\beta$ accumulates as $k$ increases, possibly even beyond $n$. The profile log likelihood for the model with $k(k+1)/2$ covariance parameters behaves anomalously in two respects. On the one hand, it is a log likelihood, so the derivatives satisfy the Bartlett identities. On the other hand, the Fisher information for $\beta$ increases to a maximum at $k=n/2$, decreasing to zero for $k\ge n$. In any parametric statistical model, one expects the Fisher information to increase with additional data; decreasing Fisher information is an anomaly demanding an explanation.