ReLU-Based and DNN-Based Generalized Maximum Score Estimators

TL;DR

This paper introduces a ReLU-based maximum score estimator that is easier to optimize and can be extended to more general models, with proven convergence rates and a DNN implementation.

Contribution

It proposes a novel ReLU-based maximum score estimator, extending the original method to a broader framework and enabling practical implementation via deep neural networks.

Findings

ReLU-based estimator is easier to optimize with gradient methods.

The estimator achieves a convergence rate of n^{-s/(2s+1)}.

The method can be implemented efficiently using DNN software.

Abstract

We propose a new formulation of the maximum score estimator that uses compositions of rectified linear unit (ReLU) functions, instead of indicator functions as in Manski (1975,1985), to encode the sign alignment restrictions. Since the ReLU function is Lipschitz, our new ReLU-based maximum score criterion function is substantially easier to optimize using standard gradient-based optimization pacakges. We also show that our ReLU-based maximum score (RMS) estimator can be generalized to an umbrella framework defined by multi-index single-crossing (MISC) conditions, while the original maximum score estimator cannot be applied. We establish the $n^{-s/(2s+1)}$ convergence rate and asymptotic normality for the RMS estimator under order-$s$ Holder smoothness. In addition, we propose an alternative estimator using a further reformulation of RMS as a special layer in a deep neural network (DNN) architecture, which allows the estimation procedure to be implemented via state-of-the-art software and hardware for DNN.