Identifying and Exploiting Structures for Reliable Deep Learning

TL;DR

This paper investigates the causes of unreliability in deep learning systems, such as adversarial vulnerability and over-confidence, and proposes efficient algorithms that exploit neural network structures to improve their robustness and trustworthiness.

Contribution

It introduces methods that leverage neural network structures to mitigate common unreliability issues in deep learning, providing practical solutions.

Findings

Identified key structures in neural networks related to unreliability.

Developed computationally efficient algorithms for robustness.

Demonstrated improved reliability in deep learning models.

Abstract

Deep learning research has recently witnessed an impressively fast-paced progress in a wide range of tasks including computer vision, natural language processing, and reinforcement learning. The extraordinary performance of these systems often gives the impression that they can be used to revolutionise our lives for the better. However, as recent works point out, these systems suffer from several issues that make them unreliable for use in the real world, including vulnerability to adversarial attacks (Szegedy et al. [248]), tendency to memorise noise (Zhang et al. [292]), being over-confident on incorrect predictions (miscalibration) (Guo et al. [99]), and unsuitability for handling private data (Gilad-Bachrach et al. [88]). In this thesis, we look at each of these issues in detail, investigate their causes, and propose computationally cheap algorithms for mitigating them in practice. To do this, we identify structures in deep neural networks that can be exploited to mitigate the above causes of unreliability of deep learning algorithms.