Square Root Compression and Noise Effects in Digitally Transformed Images

TL;DR

This paper analyzes how square-root data compression introduces systematic errors in digitized images and proposes a Bayesian decoding method to significantly reduce these errors, applicable to current and future instruments.

Contribution

The paper introduces a Bayesian decoding approach that minimizes systematic errors from nonlinear square-root coding in digitized data, applicable to various coding schemes.

Findings

Systematic bias from square root coding can be reduced by a factor of 20.

The method is effective even with loosely known noise properties.

Applicable to both new and existing data sets.

Abstract

We report on a particular example of noise and data representation interacting to introduce systematic error. Many instruments collect integer digitized values and appy nonlinear coding, in particular square-root coding, to compress the data for transfer or downlink; this can introduce surprising systematic errors when they are decoded for analysis. Square root coding and subsequent decoding typically introduces a variable, $\pm 1$ count value-dependent systematic bias in the data after reconstitution. This is significant when large numbers of measurements (e.g., image pixels) are averaged together. Using direct modeling of the probabiliity distribution of particular coded values in the presence of instrument noise, one may apply Bayes' Theorem to construct a decoding table that reduces this error source to a very small fraction of a digitizer step; in our example, systematic error from square root coding is reduced by a factor of 20 from 0.23 count RMS to 0.013 count RMS. The method is suitable both for new experiments such as the upcoming PUNCH mission, and also for post facto application to existing data sets -- even if the instrument noise properties are only loosely known. Further, the method does not depend on the specifics of the coding formula, and may be applied to other forms of nonlinear coding or representation of data values.