Task-Agnostic Experts Composition for Continual Learning

TL;DR

This paper introduces a compositional expert ensemble method for continual learning, which improves accuracy and efficiency by leveraging zero-shot expert models to decompose complex tasks.

Contribution

It presents a novel task-agnostic expert composition approach that enhances continual learning performance and efficiency compared to baseline methods.

Findings

Achieves higher accuracy than baseline algorithms.

Requires less computational resources.

Demonstrates effectiveness on a challenging compositionality benchmark.

Abstract

Compositionality is one of the fundamental abilities of the human reasoning process, that allows to decompose a complex problem into simpler elements. Such property is crucial also for neural networks, especially when aiming for a more efficient and sustainable AI framework. We propose a compositional approach by ensembling zero-shot a set of expert models, assessing our methodology using a challenging benchmark, designed to test compositionality capabilities. We show that our Expert Composition method is able to achieve a much higher accuracy than baseline algorithms while requiring less computational resources, hence being more efficient.