Collaborative likelihood-ratio estimation over graphs

TL;DR

This paper introduces a graph-based extension of likelihood-ratio estimation, enabling collaborative inference of node-specific pdfs by leveraging graph structure, with theoretical convergence guarantees and superior empirical performance.

Contribution

It proposes GRULSIF, a novel non-parametric method for collaborative likelihood-ratio estimation over graphs, with derived convergence rates and improved accuracy over existing methods.

Findings

GRULSIF outperforms state-of-the-art independent methods in experiments.

Theoretical convergence rates depend on graph size and task similarity.

Collaboration improves estimation accuracy when graph structure encodes task similarities.

Abstract

Assuming we have iid observations from two unknown probability density functions (pdfs), $p$ and $q$, the likelihood-ratio estimation (LRE) is an elegant approach to compare the two pdfs only by relying on the available data. In this paper, we introduce the first -to the best of our knowledge-graph-based extension of this problem, which reads as follows: Suppose each node $v$ of a fixed graph has access to observations coming from two unknown node-specific pdfs, $p_v$ and $q_v$, and the goal is to estimate for each node the likelihood-ratio between both pdfs by also taking into account the information provided by the graph structure. The node-level estimation tasks are supposed to exhibit similarities conveyed by the graph, which suggests that the nodes could collaborate to solve them more efficiently. We develop this idea in a concrete non-parametric method that we call Graph-based Relative Unconstrained Least-squares Importance Fitting (GRULSIF). We derive convergence rates for our collaborative approach that highlights the role played by variables such as the number of available observations per node, the size of the graph, and how accurately the graph structure encodes the similarity between tasks. These theoretical results explicit the situations where collaborative estimation effectively leads to an improvement in performance compared to solving each problem independently. Finally, in a series of experiments, we illustrate how GRULSIF infers the likelihood-ratios at the nodes of the graph more accurately compared to state-of-the art LRE methods, which would operate independently at each node, and we also verify that the behavior of GRULSIF is aligned with our previous theoretical analysis.