Guided Learning: Lubricating End-to-End Modeling for Multi-stage Decision-making

TL;DR

This paper introduces Guided Learning, a new framework that improves end-to-end multi-stage decision-making models by guiding intermediate training towards phased goals, addressing optimization challenges and enhancing performance.

Contribution

The paper proposes Guided Learning, a novel approach that uses a guide function to improve training of multi-stage decision models, overcoming collapse issues in end-to-end neural networks.

Findings

Guided Learning outperforms traditional stage-wise methods.

It significantly improves end-to-end model training stability.

Demonstrated effectiveness in quantitative investment strategies.

Abstract

Multi-stage decision-making is crucial in various real-world artificial intelligence applications, including recommendation systems, autonomous driving, and quantitative investment systems. In quantitative investment, for example, the process typically involves several sequential stages such as factor mining, alpha prediction, portfolio optimization, and sometimes order execution. While state-of-the-art end-to-end modeling aims to unify these stages into a single global framework, it faces significant challenges: (1) training such a unified neural network consisting of multiple stages between initial inputs and final outputs often leads to suboptimal solutions, or even collapse, and (2) many decision-making scenarios are not easily reducible to standard prediction problems. To overcome these challenges, we propose Guided Learning, a novel methodological framework designed to enhance end-to-end learning in multi-stage decision-making. We introduce the concept of a ``guide'', a function that induces the training of intermediate neural network layers towards some phased goals, directing gradients away from suboptimal collapse. For decision scenarios lacking explicit supervisory labels, we incorporate a utility function that quantifies the ``reward'' of the throughout decision. Additionally, we explore the connections between Guided Learning and classic machine learning paradigms such as supervised, unsupervised, semi-supervised, multi-task, and reinforcement learning. Experiments on quantitative investment strategy building demonstrate that guided learning significantly outperforms both traditional stage-wise approaches and existing end-to-end methods.