Inverse Entropic Optimal Transport Solves Semi-supervised Learning via Data Likelihood Maximization

TL;DR

This paper introduces a novel semi-supervised learning method that maximizes data likelihood by integrating paired and unpaired data, leveraging inverse entropic optimal transport, and demonstrating both theoretical recovery guarantees and empirical effectiveness.

Contribution

It develops a new semi-supervised learning framework connecting likelihood maximization with inverse entropic OT, enabling end-to-end learning of conditional distributions from mixed data.

Findings

The method can recover true conditional distributions with arbitrarily small error.

Empirical results show effective learning of conditional distributions using combined data.

The approach establishes a theoretical link between likelihood maximization and inverse entropic OT.

Abstract

Learning conditional distributions $\pi^*(\cdot|x)$ is a central problem in machine learning, which is typically approached via supervised methods with paired data $(x,y) \sim \pi^*$. However, acquiring paired data samples is often challenging, especially in problems such as domain translation. This necessitates the development of $\textit{semi-supervised}$ models that utilize both limited paired data and additional unpaired i.i.d. samples $x \sim \pi^*_x$ and $y \sim \pi^*_y$ from the marginal distributions. The usage of such combined data is complex and often relies on heuristic approaches. To tackle this issue, we propose a new learning paradigm that integrates both paired and unpaired data $\textbf{seamlessly}$ using the data likelihood maximization techniques. We demonstrate that our approach also connects intriguingly with inverse entropic optimal transport (OT). This finding allows us to apply recent advances in computational OT to establish an $\textbf{end-to-end}$ learning algorithm to get $\pi^*(\cdot|x)$. In addition, we derive the universal approximation property, demonstrating that our approach can theoretically recover true conditional distributions with arbitrarily small error. Furthermore, we demonstrate through empirical tests that our method effectively learns conditional distributions using paired and unpaired data simultaneously.