A New Learning Paradigm for Random Vector Functional-Link Network: RVFL+

TL;DR

This paper introduces RVFL+, a novel neural network model that incorporates privileged information during training, enhancing generalization and performance, especially when combined with kernel methods, as demonstrated on multiple real-world datasets.

Contribution

The paper presents RVFL+, the first to integrate the LUPI paradigm with random vector functional link networks, and extends it with kernel learning (KRVFL+), providing theoretical analysis and superior experimental results.

Findings

RVFL+ outperforms existing methods on 14 datasets.

Theoretical generalization bounds are established for RVFL+.

Kernel extension KRVFL+ enhances nonlinear feature learning.

Abstract

In school, a teacher plays an important role in various classroom teaching patterns. Likewise to this human learning activity, the learning using privileged information (LUPI) paradigm provides additional information generated by the teacher to 'teach' learning models during the training stage. Therefore, this novel learning paradigm is a typical Teacher-Student Interaction mechanism. This paper is the first to present a random vector functional link network based on the LUPI paradigm, called RVFL+. Rather than simply combining two existing approaches, the newly-derived RVFL+ fills the gap between classical randomized neural networks and the newfashioned LUPI paradigm, which offers an alternative way to train RVFL networks. Moreover, the proposed RVFL+ can perform in conjunction with the kernel trick for highly complicated nonlinear feature learning, which is termed KRVFL+. Furthermore, the statistical property of the proposed RVFL+ is investigated, and we present a sharp and high-quality generalization error bound based on the Rademacher complexity. Competitive experimental results on 14 real-world datasets illustrate the great effectiveness and efficiency of the novel RVFL+ and KRVFL+, which can achieve better generalization performance than state-of-the-art methods.