# A universal rank-order transform to extract signals from noisy data

## TL;DR

This paper presents a rank-order based, nonparametric method for extracting signals from noisy data, effectively handling outliers and heavy-tailed noise, with broad applications including nonlinear parameter estimation.

## Contribution

The paper introduces a novel rank-order transform method that is insensitive to outliers, capable of distinguishing signals from noise, and applicable to nonlinear and heavy-tailed noise scenarios.

## Key findings

- Successfully distinguishes chaos from white noise.
- Outperforms least squares in radioactive decay parameter estimation.
- Excels in trend extraction from heavy-tailed noise.

## Abstract

We introduce an ordinate method for noisy data analysis, based solely on rank information and thus insensitive to outliers. The method is nonparametric, objective, and the required data processing is parsimonious. Main ingredients are a rank-order data matrix and its transform to a stable form, which provide linear trends in excellent agreement with least squares regression, despite the loss of magnitude information. A group symmetry orthogonal decomposition of the 2D rank-order transform for iid (white) noise is further ordered by principal component analysis. This two-step procedure provides a noise "etalon" used to characterize arbitrary stationary stochastic processes. The method readily distinguishes both the Ornstein-Uhlenbeck process and chaos generated by the logistic map from white noise. Ranking within randomness differs fundamentally from that in deterministic chaos and signals, thus forming the basis for signal detection. To further illustrate the breadth of applications, we apply this ordinate method to the canonical nonlinear parameter estimation problem of two-species radioactive decay, outperforming special-purpose least square software. It is demonstrated that the method excels when extracting trends in heavy-tailed noise and, unlike the Thiele-Sen estimator, is not limited to linear regression. Lastly, a simple expression is given that yields a close approximation for signal extraction of an underlying generally nonlinear signal.