ConquerNet: Convolution-Smoothed Quantile ReLU Neural Networks with Minimax Guarantees

TL;DR

ConquerNet introduces convolution-smoothed quantile ReLU neural networks that provide smooth optimization objectives, theoretical minimax guarantees, and improved empirical performance over standard methods in distributional learning.

Contribution

It proposes a novel neural network architecture with smoothing techniques that ensure theoretical guarantees and enhanced empirical quantile regression performance.

Findings

Outperforms standard quantile neural networks in accuracy.

Provides nonasymptotic risk bounds with minimax guarantees.

Shows significant improvements at high and low quantiles.

Abstract

Quantile regression is a fundamental tool for distributional learning but poses significant optimization challenges for deep models due to the non-smoothness of the pinball loss. We propose ConquerNet, a class of \textbf{con}volution-smoothed \textbf{qu}antil\textbf{e} \textbf{R}eLU neural \textbf{net}works, which yield smooth objectives while preserving the underlying quantile structure. We establish general nonasymptotic risk bounds for ConquerNet under mild conditions, providing minimax guarantees over Besov function classes. In numerical studies, we demonstrate that the proposed approach outperforms standard quantile neural networks at multiple quantile levels, showing improved estimation accuracy and training efficiency across the board, with particularly pronounced advantages at high and low quantiles.