Estimation of Monotone Multi-Index Models

TL;DR

This paper introduces an algorithm for estimating monotone multi-index models with nonnegative index vectors, extending linear and isotonic regression, and provides theoretical guarantees on estimation accuracy.

Contribution

It proposes a novel integer programming-based algorithm for monotone multi-index model estimation with theoretical loss guarantees.

Findings

Algorithm achieves accurate estimation under monotonicity constraints

Provides theoretical bounds on the $L_2$ loss of the estimator

Generalizes linear and isotonic regression models

Abstract

In a multi-index model with $k$ index vectors, the input variables are transformed by taking inner products with the index vectors. A transfer function $f: \mathbb{R}^k \to \mathbb{R}$ is applied to these inner products to generate the output. Thus, multi-index models are a generalization of linear models. In this paper, we consider monotone multi-index models. Namely, the transfer function is assumed to be coordinate-wise monotone. The monotone multi-index model therefore generalizes both linear regression and isotonic regression, which is the estimation of a coordinate-wise monotone function. We consider the case of nonnegative index vectors. We provide an algorithm based on integer programming for the estimation of monotone multi-index models, and provide guarantees on the $L_2$ loss of the estimated function relative to the ground truth.