Forecasting in the presence of scale-free noise

TL;DR

This paper introduces a new method for forecasting signals in the presence of scale-free noise, which is common in many scientific fields, providing performance guarantees and broad applicability.

Contribution

The paper develops a novel approach to forecasting with scale-free noise, overcoming limitations of traditional methods that assume rational power spectra.

Findings

Method effectively handles scale-free noise in forecasting.

Provides theoretical performance guarantees.

Applicable to diverse fields like neuroscience and finance.

Abstract

The extraction of signals from noise is a common problem in all areas of science and engineering. A particularly useful version is that of forecasting: determining a causal filter that estimates a future value of a hidden process from past observations. Current techniques for deriving the filter require that the noise be well described by rational power spectra. However, scale-free noises, whose spectra scale as a non-integer power of frequency, are ubiquitous in practice. We establish a method, together with performance guarantees, that solves the forecasting problem in the presence of scale-free noise. Via the duality between estimation and control, our technique can be used to design control for distributed systems. These results will have wide-ranging applications in neuroscience, finance, fluid dynamics, and quantum measurements.