Effort and Substance Use: Differentiating Tobacco Use Through Reinforcement Learning of Effort Based Decision Making

TL;DR

This study uses computational models to show how substance use affects decision-making related to effort and reward, revealing differences in learning processes among users.

Contribution

The study introduces a reinforcement learning framework to differentiate tobacco use groups based on effort-based decision-making parameters.

Findings

Temporal difference reinforcement learning models best fit effort-based choice behavior.

Substance use status correlates with distinct patterns in learning rate, future discounting, and choice temperature.

Linear discriminant analysis achieved 76% accuracy in classifying substance use groups.

Abstract

Effort-based decision making evaluates rewards relative to the effort required to obtain it, an important process of healthy goal-directed motivation and behavior. Computational models provide mechanistic insights underlying choice behavior and potential alterations in neuropsychiatric disorders, including substance use disorders. We applied computational models to effort-based choice behavior to characterize underlying decision processes and if these mechanisms differ by substance use status.

Participants completed the Effort Expenditure for Rewards Task, choosing between low- and high-effort options for monetary rewards varying in magnitude and probability. Participants met criteria for no tobacco use (n = 23), current tobacco use disorder (n = 26), former tobacco use disorder (n = 22), and tobacco and opioid use disorder (n = 29). Computational models from two families, Subjective Value and Reinforcement Learning, were fit and compared. Parameters from the best-fitting model underwent principal components analysis and linear discriminant analysis.

Temporal difference reinforcement learning model demonstrated greater model evidence and predictive accuracy, indicating better fit to effort-based choice behavior. Principal components analysis revealed meaningful multivariate distinctions: PC1 differentiated all groups except individuals without tobacco use versus individuals with tobacco use disorder; PC3 distinguished tobacco and opioid use disorder from all other groups. Linear discriminant analysis demonstrated group separation with 76% classification accuracy.

A reinforcement learning framework better explained participants’ effort-based choice behavior. Substance use status relates to dynamic behavioral changes (i.e. learning) as measured by the multivariate combination of learning rate, future discounting, and choice temperature.

Figures