Generalization Through the Lens of Learning Dynamics

TL;DR

This paper explores how learning dynamics influence the ability of deep neural networks, in supervised and reinforcement learning, to generalize effectively to new, unseen situations, addressing a key challenge in deploying reliable AI systems.

Contribution

It offers new insights into the role of learning dynamics in neural network generalization, bridging gaps in understanding for supervised and reinforcement learning.

Findings

Learning dynamics significantly impact generalization performance.

Deep neural networks can generalize well despite theoretical challenges.

Insights help improve the reliability of AI systems in real-world applications.

Abstract

A machine learning (ML) system must learn not only to match the output of a target function on a training set, but also to generalize to novel situations in order to yield accurate predictions at deployment. In most practical applications, the user cannot exhaustively enumerate every possible input to the model; strong generalization performance is therefore crucial to the development of ML systems which are performant and reliable enough to be deployed in the real world. While generalization is well-understood theoretically in a number of hypothesis classes, the impressive generalization performance of deep neural networks has stymied theoreticians. In deep reinforcement learning (RL), our understanding of generalization is further complicated by the conflict between generalization and stability in widely-used RL algorithms. This thesis will provide insight into generalization by studying the learning dynamics of deep neural networks in both supervised and reinforcement learning tasks.