Learning by Self-Explanation, with Application to Neural Architecture Search

TL;DR

This paper introduces LeaSE, a novel machine learning approach inspired by human self-explanation, which enhances learning by iteratively explaining and re-learning, and demonstrates its effectiveness in neural architecture search tasks.

Contribution

The paper proposes LeaSE, a four-level optimization framework for self-explanation in machine learning, and applies it successfully to neural architecture search.

Findings

LeaSE improves neural architecture search performance.

Experimental results on CIFAR and ImageNet validate LeaSE's effectiveness.

The method outperforms existing approaches in accuracy and efficiency.

Abstract

Learning by self-explanation is an effective learning technique in human learning, where students explain a learned topic to themselves for deepening their understanding of this topic. It is interesting to investigate whether this explanation-driven learning methodology broadly used by humans is helpful for improving machine learning as well. Based on this inspiration, we propose a novel machine learning method called learning by self-explanation (LeaSE). In our approach, an explainer model improves its learning ability by trying to clearly explain to an audience model regarding how a prediction outcome is made. LeaSE is formulated as a four-level optimization problem involving a sequence of four learning stages which are conducted end-to-end in a unified framework: 1) explainer learns; 2) explainer explains; 3) audience learns; 4) explainer re-learns based on the performance of the audience. We develop an efficient algorithm to solve the LeaSE problem. We apply LeaSE for neural architecture search on CIFAR-100, CIFAR-10, and ImageNet. Experimental results strongly demonstrate the effectiveness of our method.