Parameters not empirically identifiable or distinguishable, including correlation between Gaussian observations

TL;DR

The paper demonstrates that certain parameters, including Gaussian correlation, are theoretically identifiable but cannot be empirically estimated or distinguished, highlighting fundamental limitations in statistical inference.

Contribution

It introduces a framework distinguishing theoretical identifiability from empirical distinguishability and shows that some parameters, like Gaussian correlation, cannot be empirically identified or distinguished.

Findings

Constant Gaussian correlations cannot be empirically distinguished.

Some parameters are theoretically identifiable but not empirically estimable.

Conditions are provided for distinguishing independence from dependence.

Abstract

Note: Accepted version, published in Statistical Papers, https://doi.org/10.1007/s00362-023-01414-3. It is shown that some theoretically identifiable parameters cannot be empirically identified, meaning that no consistent estimator of them can exist. An important example is a constant correlation between Gaussian observations (in presence of such correlation not even the mean can be empirically identified). Empirical identifiability and three versions of empirical distinguishability are defined. Two different constant correlations between Gaussian observations cannot even be empirically distinguished. A further example are cluster membership parameters in $k$-means clustering. Several existing results in the literature are connected to the new framework. General conditions are discussed under which independence can be distinguished from dependence.