# TokenVerse++: Towards Flexible Multitask Learning with Dynamic Task Activation

## TL;DR

TokenVerse++ enhances multitask learning by enabling dynamic task activation through learnable vectors, allowing training with partially labeled data and improving performance across multiple tasks.

## Contribution

It introduces learnable vectors for dynamic task activation, allowing training with partial labels and improving multitask learning efficiency.

## Key findings

- Successfully integrated partial labeled datasets for ASR and language ID
- Achieved comparable or better results than TokenVerse on multiple tasks
- Demonstrated practical advantages of dynamic task activation

## Abstract

Token-based multitasking frameworks like TokenVerse require all training utterances to have labels for all tasks, hindering their ability to leverage partially annotated datasets and scale effectively. We propose TokenVerse++, which introduces learnable vectors in the acoustic embedding space of the XLSR-Transducer ASR model for dynamic task activation. This core mechanism enables training with utterances labeled for only a subset of tasks, a key advantage over TokenVerse. We demonstrate this by successfully integrating a dataset with partial labels, specifically for ASR and an additional task, language identification, improving overall performance. TokenVerse++ achieves results on par with or exceeding TokenVerse across multiple tasks, establishing it as a more practical multitask alternative without sacrificing ASR performance.