On the Success Probability of the Box-Constrained Rounding and Babai Detectors

TL;DR

This paper analyzes the success probabilities of box-constrained rounding and Babai detectors in linear models, revealing conditions under which the rounding detector can outperform the Babai detector.

Contribution

It derives formulas for success probabilities in different scenarios and proves that the rounding detector can have higher success probability when the parameter vector is deterministic.

Findings

Formulas for success probabilities of both detectors in two scenarios.

Rounding detector can outperform Babai detector when the vector is deterministic.

Always holds that the success probability of rounding is less than or equal to Babai when the vector is uniformly distributed.

Abstract

In communications, one frequently needs to detect a parameter vector $\hbx$ in a box from a linear model. The box-constrained rounding detector $\x^\sBR$ and Babai detector $\x^\sBB$ are often used to detect $\hbx$ due to their high probability of correct detection, which is referred to as success probability, and their high efficiency of implimentation. It is generally believed that the success probability $P^\sBR$ of $\x^\sBR$ is not larger than the success probability $P^\sBB$ of $\x^\sBB$. In this paper, we first present formulas for $P^\sBR$ and $P^\sBB$ for two different situations: $\hbx$ is deterministic and $\hbx$ is uniformly distributed over the constraint box. Then, we give a simple example to show that $P^\sBR$ may be strictly larger than $P^\sBB$ if $\hbx$ is deterministic, while we rigorously show that $P^\sBR\leq P^\sBB$ always holds if $\hbx$ is uniformly distributed over the constraint box.