Contextual Preference Distribution Learning

TL;DR

This paper introduces a sequential learning pipeline that models human preference distributions conditioned on context, improving decision-making under uncertainty in risk-averse settings, demonstrated in a ridesharing simulation.

Contribution

It develops a novel method to learn and generate preference distributions conditioned on context, surpassing existing inverse optimization techniques in capturing shifts for risk-averse decisions.

Findings

Reduces average post-decision surprise significantly in simulations.

Outperforms risk-neutral and risk-averse baselines.

Provides a scalable approach for context-dependent preference modeling.

Abstract

Decision-making problems often feature uncertainty stemming from heterogeneous and context-dependent human preferences. To address this, we propose a sequential learning-and-optimization pipeline to learn preference distributions and leverage them to solve downstream problems, for example risk-averse formulations. We focus on human choice settings that can be formulated as (integer) linear programs. In such settings, existing inverse optimization and choice modelling methods infer preferences from observed choices but typically produce point estimates or fail to capture contextual shifts, making them unsuitable for risk-averse decision-making. Using a bounded-variance score function gradient estimator, we train a predictive model mapping contextual features to a rich class of parameterizable distributions. This approach yields a maximum likelihood estimate. The model generates scenarios for unseen contexts in the subsequent optimization phase. In a synthetic ridesharing environment, our approach reduces average post-decision surprise by up to 114$\times$ compared to a risk-neutral approach with perfect predictions and up to 25$\times$ compared to leading risk-averse baselines.