Deep neural networks for choice analysis: Enhancing behavioral regularity with gradient regularization

TL;DR

This paper introduces a gradient regularization framework to improve the behavioral regularity of deep neural networks in travel choice modeling, balancing predictive accuracy with demand law compliance.

Contribution

It proposes novel metrics for behavioral regularity and a constrained optimization framework with six gradient regularizers to enhance DNNs' demand law adherence.

Findings

Gradient regularization increases behavioral regularity by around 6 percentage points.

In small samples, it improves regularity by about 20 pp and log-likelihood by 1.7%.

Out-of-domain generalization benefits significantly from regularization, improving poor models by around 65 pp.

Abstract

Deep neural networks (DNNs) frequently present behaviorally irregular patterns, significantly limiting their practical potentials and theoretical validity in travel behavior modeling. This study proposes strong and weak behavioral regularities as novel metrics to evaluate the monotonicity of individual demand functions (known as the "law of demand"), and further designs a constrained optimization framework with six gradient regularizers to enhance DNNs' behavioral regularity. The proposed framework is applied to travel survey data from Chicago and London to examine the trade-off between predictive power and behavioral regularity for large vs. small sample scenarios and in-domain vs. out-of-domain generalizations. The results demonstrate that, unlike models with strong behavioral foundations such as the multinomial logit, the benchmark DNNs cannot guarantee behavioral regularity. However, gradient regularization (GR) increases DNNs' behavioral regularity by around 6 percentage points (pp) while retaining their relatively high predictive power. In the small sample scenario, GR is more effective than in the large sample scenario, simultaneously improving behavioral regularity by about 20 pp and log-likelihood by around 1.7%. Comparing with the in-domain generalization of DNNs, GR works more effectively in out-of-domain generalization: it drastically improves the behavioral regularity of poorly performing benchmark DNNs by around 65 pp, indicating the criticality of behavioral regularization for enhancing model transferability and application in forecasting. Moreover, the proposed framework is applicable to other NN-based choice models such as TasteNets. Future studies could use behavioral regularity as a metric along with log-likelihood in evaluating travel demand models, and investigate other methods to further enhance behavioral regularity when adopting complex machine learning models.