Semi-Supervised Single- and Multi-Domain Regression with Multi-Domain Training

TL;DR

This paper introduces a Bayesian-based semi-supervised approach for single- and multi-domain regression that effectively utilizes unpaired labeled and large unlabeled datasets, with applications in facial image editing and audio-visual recognition.

Contribution

It formulates a worst-case design strategy for multi-domain regression with unpaired data, explicitly considering labeled set sizes and special cases, advancing multi-modal learning techniques.

Findings

Effective in facial expression removal

Improves audio-visual word recognition accuracy

Outperforms several recent multi-modal algorithms

Abstract

We address the problems of multi-domain and single-domain regression based on distinct and unpaired labeled training sets for each of the domains and a large unlabeled training set from all domains. We formulate these problems as a Bayesian estimation with partial knowledge of statistical relations. We propose a worst-case design strategy and study the resulting estimators. Our analysis explicitly accounts for the cardinality of the labeled sets and includes the special cases in which one of the labeled sets is very large or, in the other extreme, completely missing. We demonstrate our estimators in the context of removing expressions from facial images and in the context of audio-visual word recognition, and provide comparisons to several recently proposed multi-modal learning algorithms.