Forward Stagewise Additive Model for Collaborative Multiview Boosting

TL;DR

This paper introduces a mathematically grounded multiview collaborative boosting framework for multiclass classification, utilizing a novel exponential loss function to improve convergence and reduce overfitting.

Contribution

It formulates a new multiview boosting algorithm based on a forward stagewise additive model with a novel exponential loss, extending to multiclass problems and providing theoretical analysis.

Findings

Converges closer to the global minimum in exponential loss space.

Achieves lower error bounds and higher margins than previous models.

Outperforms traditional and recent multiview boosting algorithms.

Abstract

Multiview assisted learning has gained significant attention in recent years in supervised learning genre. Availability of high performance computing devices enables learning algorithms to search simultaneously over multiple views or feature spaces to obtain an optimum classification performance. The paper is a pioneering attempt of formulating a mathematical foundation for realizing a multiview aided collaborative boosting architecture for multiclass classification. Most of the present algorithms apply multiview learning heuristically without exploring the fundamental mathematical changes imposed on traditional boosting. Also, most of the algorithms are restricted to two class or view setting. Our proposed mathematical framework enables collaborative boosting across any finite dimensional view spaces for multiclass learning. The boosting framework is based on forward stagewise additive model which minimizes a novel exponential loss function. We show that the exponential loss function essentially captures difficulty of a training sample space instead of the traditional `1/0' loss. The new algorithm restricts a weak view from over learning and thereby preventing overfitting. The model is inspired by our earlier attempt on collaborative boosting which was devoid of mathematical justification. The proposed algorithm is shown to converge much nearer to global minimum in the exponential loss space and thus supersedes our previous algorithm. The paper also presents analytical and numerical analysis of convergence and margin bounds for multiview boosting algorithms and we show that our proposed ensemble learning manifests lower error bound and higher margin compared to our previous model. Also, the proposed model is compared with traditional boosting and recent multiview boosting algorithms.