Learning Set Functions with Implicit Differentiation

TL;DR

This paper presents an efficient implicit differentiation method for learning set functions from data, improving computational feasibility in fixed-point iteration-based models for applications like recommendation and anomaly detection.

Contribution

It introduces an implicit differentiation approach to address computational challenges in fixed-point iteration methods for learning set functions, with convergence analysis.

Findings

Method reduces computational cost of backpropagation

Effective on synthetic and real-world subset selection tasks

Demonstrates improved efficiency over previous approaches

Abstract

Ou et al. (2022) introduce the problem of learning set functions from data generated by a so-called optimal subset oracle. Their approach approximates the underlying utility function with an energy-based model, whose parameters are estimated via mean-field variational inference. Ou et al. (2022) show this reduces to fixed point iterations; however, as the number of iterations increases, automatic differentiation quickly becomes computationally prohibitive due to the size of the Jacobians that are stacked during backpropagation. We address this challenge with implicit differentiation and examine the convergence conditions for the fixed-point iterations. We empirically demonstrate the efficiency of our method on synthetic and real-world subset selection applications including product recommendation, set anomaly detection and compound selection tasks.