Generalization-Memorization Machines

TL;DR

This paper introduces a generalization-memorization mechanism for machine learning that enhances memorization of training data without overfitting, leading to the development of Generalization-Memorization Machines (GMM) with efficient quadratic programming solutions.

Contribution

The paper proposes a novel generalization-memorization mechanism and introduces GMM, unifying and extending previous kernel methods with improved memorization and generalization capabilities.

Findings

GMM effectively balances memorization and generalization.

GMM's quadratic programming problems are computationally efficient.

Experimental results confirm GMM's superior performance on training data.

Abstract

Classifying the training data correctly without over-fitting is one of the goals in machine learning. In this paper, we propose a generalization-memorization mechanism, including a generalization-memorization decision and a memory modeling principle. Under this mechanism, error-based learning machines improve their memorization abilities of training data without over-fitting. Specifically, the generalization-memorization machines (GMM) are proposed by applying this mechanism. The optimization problems in GMM are quadratic programming problems and could be solved efficiently. It should be noted that the recently proposed generalization-memorization kernel and the corresponding support vector machines are the special cases of our GMM. Experimental results show the effectiveness of the proposed GMM both on memorization and generalization.