Learning Monotonic Probabilities with a Generative Cost Model

TL;DR

This paper introduces the Generative Cost Model (GCM) and Implicit GCM (IGCM), novel approaches for modeling monotonic relationships in machine learning by focusing on latent cost variables, outperforming existing methods.

Contribution

The paper proposes a new generative modeling framework for strict and implicit monotonicity, transforming the problem into latent cost variable modeling.

Findings

Outperforms existing monotonic modeling techniques

Validated through numerical simulation and experiments on public datasets

Provides a generative approach to enforce monotonicity constraints

Abstract

In many machine learning tasks, it is often necessary for the relationship between input and output variables to be monotonic, including both strictly monotonic and implicitly monotonic relationships. Traditional methods for maintaining monotonicity mainly rely on construction or regularization techniques, whereas this paper shows that the issue of strict monotonic probability can be viewed as a partial order between an observable revenue variable and a latent cost variable. This perspective enables us to reformulate the monotonicity challenge into modeling the latent cost variable. To tackle this, we introduce a generative network for the latent cost variable, termed the Generative Cost Model (GCM), which inherently addresses the strict monotonic problem, and propose the Implicit Generative Cost Model (IGCM) to address the implicit monotonic problem. We further validate our approach with a numerical simulation of quantile regression and conduct multiple experiments on public datasets, showing that our method significantly outperforms existing monotonic modeling techniques. The code for our experiments can be found at https://github.com/tyxaaron/GCM.