Learning State Representations for Query Optimization with Deep Reinforcement Learning

TL;DR

This paper explores using deep reinforcement learning to develop state representations for query optimization, aiming to improve the process by learning how subqueries evolve through actions.

Contribution

It introduces a novel approach to encoding subquery properties with learned representations and discusses how to utilize these for reinforcement learning-based query optimization.

Findings

Preliminary results demonstrate the feasibility of the approach.

The proposed state representation captures subquery properties effectively.

Discussion on how to leverage state representations for optimization improvement.

Abstract

Deep reinforcement learning is quickly changing the field of artificial intelligence. These models are able to capture a high level understanding of their environment, enabling them to learn difficult dynamic tasks in a variety of domains. In the database field, query optimization remains a difficult problem. Our goal in this work is to explore the capabilities of deep reinforcement learning in the context of query optimization. At each state, we build queries incrementally and encode properties of subqueries through a learned representation. The challenge here lies in the formation of the state transition function, which defines how the current subquery state combines with the next query operation (action) to yield the next state. As a first step in this direction, we focus the state representation problem and the formation of the state transition function. We describe our approach and show preliminary results. We further discuss how we can use the state representation to improve query optimization using reinforcement learning.