On regret bounds for continual single-index learning

TL;DR

This paper introduces a continual learning approach for single-index models, proposing a randomized strategy that learns a shared meta-parameter across tasks with theoretical regret bounds, enabling effective transfer in an online setting.

Contribution

It extends single-index models to continual learning, proposing a novel randomized strategy with theoretical regret guarantees for online multi-task learning.

Findings

Regret bounds are established under various loss function assumptions.

The strategy effectively transfers information across tasks in an online setting.

The approach demonstrates theoretical robustness in continual single-index learning.

Abstract

In this paper, we generalize the problem of single-index model to the context of continual learning in which a learner is challenged with a sequence of tasks one by one and the dataset of each task is revealed in an online fashion. We propose a randomized strategy that is able to learn a common single-index (meta-parameter) for all tasks and a specific link function for each task. The common single-index allows to transfer the information gained from the previous tasks to a new one. We provide a rigorous theoretical analysis of our proposed strategy by proving some regret bounds under different assumption on the loss function.