Effects of the LLL reduction on the success probability of the Babai point and on the complexity of sphere decoding

TL;DR

This paper proves that LLL reduction improves the success probability of the Babai point and reduces the complexity of sphere decoding in integer least squares problems, enhancing efficiency and reliability.

Contribution

It rigorously demonstrates that LLL reduction increases Babai point success probability and decreases sphere decoding complexity, providing theoretical guarantees.

Findings

LLL reduction increases Babai point success probability

Applying LLL reduces sphere decoding complexity

A sharper lower bound for Babai point success probability is established

Abstract

The common method to estimate an unknown integer parameter vector in a linear model is to solve an integer least squares (ILS) problem. A typical approach to solving an ILS problem is sphere decoding. To make a sphere decoder faster, the well-known LLL reduction is often used as preprocessing. The Babai point produced by the Babai nearest plan algorithm is a suboptimal solution of the ILS problem. First we prove that the success probability of the Babai point as a lower bound on the success probability of the ILS estimator is sharper than the lower bound given by Hassibi and Boyd [1]. Then we show rigorously that applying the LLL reduction algorithm will increase the success probability of the Babai point. Finally we show rigorously that applying the LLL reduction algorithm will also reduce the computational complexity of sphere decoders, which is measured approximately by the number of nodes in the search tree in the literature