Injecting Text and Cross-lingual Supervision in Few-shot Learning from Self-Supervised Models

TL;DR

This paper explores leveraging cross-lingual supervision and target-language text to enhance few-shot learning in self-supervised models for low-resource languages, demonstrating significant performance improvements.

Contribution

It introduces methods to incorporate cross-lingual supervision and target-language text into self-supervised models for better transfer to new languages.

Findings

Significant improvement in few-shot learning for low-resource languages

Effective use of cross-lingual supervision in self-supervised models

Enhanced fine-tuning with target-language text and LF-MMI

Abstract

Self-supervised model pre-training has recently garnered significant interest, but relatively few efforts have explored using additional resources in fine-tuning these models. We demonstrate how universal phoneset acoustic models can leverage cross-lingual supervision to improve transfer of pretrained self-supervised representations to new languages. We also show how target-language text can be used to enable and improve fine-tuning with the lattice-free maximum mutual information (LF-MMI) objective. In three low-resource languages these techniques greatly improved few-shot learning performance.