MODL: Multilearner Online Deep Learning

TL;DR

This paper introduces MODL, a hybrid multilearner online deep learning framework that combines fast recursive logistic regression with deep learners, achieving state-of-the-art results in online data stream tasks.

Contribution

It presents a novel multilearner approach integrating fast recursive logistic regression with deep learning, improving online learning efficiency and performance.

Findings

Achieves state-of-the-art performance on online datasets

Combines fast logistic regression with deep learners effectively

Demonstrates the benefits of hybrid multilearner design

Abstract

Online deep learning tackles the challenge of learning from data streams by balancing two competing goals: fast learning and deep learning. However, existing research primarily emphasizes deep learning solutions, which are more adept at handling the ``deep'' aspect than the ``fast'' aspect of online learning. In this work, we introduce an alternative paradigm through a hybrid multilearner approach. We begin by developing a fast online logistic regression learner, which operates without relying on backpropagation. It leverages closed-form recursive updates of model parameters, efficiently addressing the fast learning component of the online learning challenge. This approach is further integrated with a cascaded multilearner design, where shallow and deep learners are co-trained in a cooperative, synergistic manner to solve the online learning problem. We demonstrate that this approach achieves state-of-the-art performance on standard online learning datasets. We make our code available: https://github.com/AntonValk/MODL