Structural Incompatibility of Differentiable Sorting and Within-Vector Rank Normalization

TL;DR

This paper demonstrates fundamental incompatibilities between differentiable sorting methods and within-vector rank normalization, providing formal conditions and characterizations that explain why certain relaxations violate key invariance and stability properties.

Contribution

It formalizes the conditions for admissibility of differentiable sorting operators and characterizes the class of operators that satisfy these conditions, highlighting inherent limitations.

Findings

SoftSort violates monotone invariance (C1) depending on temperature and input scale.

SinkhornSort violates batch independence (C2), leading to batch-dependent outputs.

Admissible operators must factor through the rank representation via a Lipschitz function.

Abstract

We show that differentiable sorting and ranking operators are structurally incompatible with within-vector rank normalization. We formalize admissibility through monotone invariance (C1), batch independence (C2), and a rank-space stability condition (C3). Gap-sensitive relaxations such as SoftSort violate (C1) by a quantitative margin that depends on the temperature and input scale. Batchwise rank relaxations such as SinkhornSort violate (C2): the same sample can be assigned outputs arbitrarily close to 0 or 1 depending solely on batch context. Condition (C3) implies (C1) under the rank representation used here and should not be read as a third independent failure mode. We also characterize the admissible class: any admissible operator must factor through the rank representation via a Lipschitz function.