Memory Constraint Online Multitask Classification

TL;DR

This paper introduces new memory-efficient online kernel algorithms for multitask classification, balancing learning from multiple tasks with fixed memory constraints, and demonstrates their effectiveness on real-world problems.

Contribution

The paper presents two novel projection-based algorithms and theoretical budget algorithms that effectively manage memory while learning multiple tasks simultaneously.

Findings

Algorithms handle high number of tasks with fixed memory.

Proposed methods outperform traditional approaches in experiments.

Sharing memory among tasks is automatically optimized.

Abstract

We investigate online kernel algorithms which simultaneously process multiple classification tasks while a fixed constraint is imposed on the size of their active sets. We focus in particular on the design of algorithms that can efficiently deal with problems where the number of tasks is extremely high and the task data are large scale. Two new projection-based algorithms are introduced to efficiently tackle those issues while presenting different trade offs on how the available memory is managed with respect to the prior information about the learning tasks. Theoretically sound budget algorithms are devised by coupling the Randomized Budget Perceptron and the Forgetron algorithms with the multitask kernel. We show how the two seemingly contrasting properties of learning from multiple tasks and keeping a constant memory footprint can be balanced, and how the sharing of the available space among different tasks is automatically taken care of. We propose and discuss new insights on the multitask kernel. Experiments show that online kernel multitask algorithms running on a budget can efficiently tackle real world learning problems involving multiple tasks.