A Practical Distributed ADMM Solver for Billion-Scale Generalized Assignment Problems

TL;DR

This paper introduces a scalable distributed solver based on BADMM for large-scale generalized assignment problems involving billions of decision variables, applicable to various real-world matching tasks.

Contribution

It develops a novel BADMM-based method that efficiently solves billion-scale assignment problems with convex objectives and separable constraints, enabling practical large-scale applications.

Findings

The method scales to hundreds of millions of items with tens of owners.

Experimental results show high accuracy on synthetic and real datasets.

The approach achieves efficient parallelization using MapReduce-like frameworks.

Abstract

Assigning items to owners is a common problem found in various real-world applications, for example, audience-channel matching in marketing campaigns, borrower-lender matching in loan management, and shopper-merchant matching in e-commerce. Given an objective and multiple constraints, an assignment problem can be formulated as a constrained optimization problem. Such assignment problems are usually NP-hard, so when the number of items or the number of owners is large, solving for exact solutions becomes challenging. In this paper, we are interested in solving constrained assignment problems with hundreds of millions of items. Thus, with just tens of owners, the number of decision variables is at billion-scale. This scale is usually seen in the internet industry, which makes decisions for large groups of users. We relax the possible integer constraint, and formulate a general optimization problem that covers commonly seen assignment problems. Its objective function is convex. Its constraints are either linear, or convex and separable by items. We study to solve our generalized assignment problems in the Bregman Alternating Direction Method of Multipliers (BADMM) framework where we exploit Bregman divergence to transform the Augmented Lagrangian into a separable form, and solve many subproblems in parallel. The entire solution can thus be implemented using a MapReduce-style distributed computation framework. We present experiment results on both synthetic and real-world datasets to verify its accuracy and scalability.