In-Context Positive-Unlabeled Learning

TL;DR

This paper introduces PUICL, a pretrained transformer that performs positive-unlabeled learning through in-context learning, eliminating the need for dataset-specific training or tuning.

Contribution

PUICL is the first transformer-based model to solve PU classification via in-context learning, trained on synthetic datasets to handle various feature-label relationships.

Findings

PUICL outperforms four standard PU baselines in AUC and accuracy.

PUICL is competitive on F1-score across 20 semi-synthetic benchmarks.

The approach demonstrates in-context learning extends to semi-supervised tabular classification.

Abstract

Positive-unlabeled (PU) learning addresses binary classification when only a set of labeled positives is available alongside a pool of unlabeled samples drawn from a mixture of positives and negatives. Existing PU methods typically require dataset-specific training or iterative optimization, which limits their applicability when many tasks must be solved quickly or with little tuning. We introduce PUICL, a pretrained transformer that solves PU classification entirely through in-context learning. PUICL is pretrained on synthetic PU datasets generated from randomly instantiated structural causal models, exposing it to a wide range of feature-label relationships and class-prior configurations. At inference time, PUICL receives the labeled positives and the unlabeled samples as a single input and returns class probabilities for the unlabeled rows in one forward pass, with no gradient updates or per-task fitting. On 20 semi-synthetic PU benchmarks derived from the UCI Machine Learning Repository, OpenML, and scikit-learn, PUICL outperforms four standard PU learning baselines in average AUC and accuracy, and is competitive on F1-score. These results show that the in-context learning paradigm extends naturally beyond fully supervised tabular prediction to the semi-supervised PU setting.