Reinforcement Learning for Stock Transactions

TL;DR

This paper explores applying reinforcement learning techniques, including Q-Learning and deep Q-Learning, to optimize stock trading decisions by modeling the problem as a Markov Decision Process and comparing different agents' performance.

Contribution

It introduces a novel RL-based framework for stock trading that defines a custom MDP and compares multiple RL agents and machine learning models for profit maximization.

Findings

RL agents can learn effective trading policies

Deep Q-Learning outperforms simpler RL methods

Machine learning models provide complementary price predictions

Abstract

Much research has been done to analyze the stock market. After all, if one can determine a pattern in the chaotic frenzy of transactions, then they could make a hefty profit from capitalizing on these insights. As such, the goal of our project was to apply reinforcement learning (RL) to determine the best time to buy a stock within a given time frame. With only a few adjustments, our model can be extended to identify the best time to sell a stock as well. In order to use the format of free, real-world data to train the model, we define our own Markov Decision Process (MDP) problem. These two papers [5] [6] helped us in formulating the state space and the reward system of our MDP problem. We train a series of agents using Q-Learning, Q-Learning with linear function approximation, and deep Q-Learning. In addition, we try to predict the stock prices using machine learning regression and classification models. We then compare our agents to see if they converge on a policy, and if so, which one learned the best policy to maximize profit on the stock market.