Towards Universal Neural Likelihood Inference

TL;DR

This paper presents UNLI, a universal neural likelihood inference framework capable of providing accurate, data-grounded likelihood predictions across diverse tasks and domains, using the ASPIRE model trained on extensive datasets.

Contribution

It introduces UNLI and the ASPIRE model, enabling zero-shot, multi-domain likelihood inference with a novel set-based reasoning approach for heterogeneous tabular data.

Findings

Achieves 15% higher F1 scores than existing models.

Reduces RMSE by 85% in zero-shot settings.

Demonstrates effective open-world active feature acquisition.

Abstract

We introduce universal neural likelihood inference (UNLI): enabling a single model to provide data-grounded, conditional likelihood predictions for arbitrary targets given any collection of observed features, across diverse domains and tasks. To achieve UNLI over heterogeneous tabular data, we develop the Arbitrary Set-based Permutation-Invariant Reasoning Engine (ASPIRE) model. Our design addresses critical gaps in existing approaches to merge semantic-understanding capabilities and generalised numerical feature reasoning within a zero-shot capable framework. Trained on over 1,400 real diverse datasets spanning various domains, ASPIRE achieves 15\% higher F1 scores and 85\% lower RMSE than existing tabular foundation models in zero-shot and few-shot settings. Lastly, this work introduces open-world active feature acquisition, where we leverage the UNLI capabilities of ASPIRE to adeptly determine next feature-values to observe to improve inference time prediction accuracies.