# A model of discrete choice based on reinforcement learning under short-term memory

## TL;DR

This paper introduces a discrete choice model based on reinforcement learning with short-term memory, revealing deviations from classical utility axioms and applying it to insurance demand analysis.

## Contribution

It develops a novel reinforcement learning-based discrete choice model incorporating short-term memory, showing deviations from Luce's axiom and utility theory axioms.

## Key findings

- Choice probabilities deviate from Luce's Choice Axiom
- Preferences violate transitivity and independence axioms
- Model explains risk behaviors in insurance demand

## Abstract

A family of models of individual discrete choice are constructed by means of statistical averaging of choices made by a subject in a reinforcement learning process, where the subject has short, k-term memory span. The choice probabilities in these models combine in a non-trivial, non-linear way the initial learning bias and the experience gained through learning. The properties of such models are discussed and, in particular, it is shown that probabilities deviate from Luce's Choice Axiom, even if the initial bias adheres to it. Moreover, we shown that the latter property is recovered as the memory span becomes large. Two applications in utility theory are considered. In the first, we use the discrete choice model to generate binary preference relation on simple lotteries. We show that the preferences violate transitivity and independence axioms of expected utility theory. Furthermore, we establish the dependence of the preferences on frames, with risk aversion for gains, and risk seeking for losses. Based on these findings we propose next a parametric model of choice based on the probability maximization principle, as a model for deviations from expected utility principle. To illustrate the approach we apply it to the classical problem of demand for insurance.