On-line learning of non-monotonic rules by simple perceptron

TL;DR

This paper investigates how a simple perceptron learns non-monotonic rules in an online setting, analyzing its generalization performance and proposing strategies to minimize errors.

Contribution

It introduces new learning strategies for perceptrons to effectively learn non-monotonic rules and estimates their asymptotic generalization error.

Findings

Perceptron can learn non-monotonic rules in an online setting.

Proposed strategies improve generalization error close to theoretical bounds.

Asymptotic analysis provides insights into learning dynamics for unlearnable rules.

Abstract

We study the generalization ability of a simple perceptron which learns unlearnable rules. The rules are presented by a teacher perceptron with a non-monotonic transfer function. The student is trained in the on-line mode. The asymptotic behaviour of the generalization error is estimated under various conditions. Several learning strategies are proposed and improved to obtain the theoretical lower bound of the generalization error.