Reciprocity-driven Sparse Network Formation

TL;DR

This paper introduces decentralized algorithms for resource exchange networks that optimize sparsity and reciprocity, using convex programming and reweighted l1 minimization, addressing NP-hardness in network formation.

Contribution

It proposes novel decentralized algorithms based on convex programming and reweighted l1 minimization to approximate sparsest reciprocal exchanges in resource networks.

Findings

Algorithms effectively balance sparsity and reciprocity.

The induced graphs exhibit desirable structural properties.

Trade-offs between sparsity and reciprocity are characterized.

Abstract

A resource exchange network is considered, where exchanges among nodes are based on reciprocity. Peers receive from the network an amount of resources commensurate with their contribution. We assume the network is fully connected, and impose sparsity constraints on peer interactions. Finding the sparsest exchanges that achieve a desired level of reciprocity is in general NP-hard. To capture near-optimal allocations, we introduce variants of the Eisenberg-Gale convex program with sparsity penalties. We derive decentralized algorithms, whereby peers approximately compute the sparsest allocations, by reweighted l1 minimization. The algorithms implement new proportional-response dynamics, with nonlinear pricing. The trade-off between sparsity and reciprocity and the properties of graphs induced by sparse exchanges are examined.

Equations

$$\text{(PEER)}\underset{{\mathbf{x}}_i\ge 0}{\max }{r}_i({\mathbf{x}}_{-i})\text{mbox}subjectto\sum _{j\ne i}{x}_{ji}=a_i.$$

$$\underset{\mathbf{x}}{\max }\sum _i{a}_i\log r_i\text{subject to}\sum _{j\ne i}{x}_{ji}=a_i,\forall i.$$

$$x_{ij}>0\text{mbox}ifandonlyif{\rho }_j=\underset{k\in {\mathcal{N}}_i}{\min }{\rho }_k,$$

$$L(\mathbf{x},\mathbf{\rho })=\sum _i{a}_i\log r_i+\sum _i{\rho }_i(a_i-\sum _{j\ne i}{x}_{ji}).$$

$$x_{ij}=0\text{mbox}and\frac{\partial L}{\partial x_{ij}}<0\text{in which case}\frac{a_i}{r_i}<{\rho }_j,$$

$$x_{ij}>0\text{mbox}and\frac{\partial L}{\partial x_{ij}}=0\text{in which case}\frac{a_i}{r_i}={\rho }_j.$$

$${\rho }_j=\frac{r_j}{a_j},\forall j.$$

$$\text{(}P0)\underset{\mathbf{x}}{\min }||\mathbf{x}|{|}_0\text{s. t.}\sum _{j\ne i}{x}_{ji}=a_i,\frac{r_i}{a_i}\ge \theta \forall i.$$

$$\text{mathds}1\left\{|x|\ne 0\right\}=\underset{ϵ\downarrow 0}{\lim }\frac{\log (1+|x|/ϵ)}{\log (1+1/ϵ)},$$

$$\underset{\mathbf{x}}{\min }\ell (\mathbf{x})\text{s. t.}\sum _{j\ne i}{x}_{ji}=a_i,\frac{r_i}{a_i}\ge \theta \forall i.$$

$$\underset{\mathbf{x}}{\min }{\mathbf{x}}^T\nabla \ell (\mathbf{x}(t))\text{s. t.}\sum _{j\ne i}{x}_{ji}=a_i,\frac{r_i}{a_i}\ge \theta \forall i,$$

$$q(x,\bar{x}):=\left\{\begin{array}[]{ll}x^{2}/(2\delta(\epsilon+\delta)),&0\leq x\leq\delta\\ x^{2}/(2\bar{x}(\epsilon+\bar{x})),&x>\delta,\end{array}\right.$$

$$\underset{\mathbf{x}}{\min }\sum _{i,j}q(x_{ij},x_{ij}(t))\text{s. t.}\sum _{j\ne i}{x}_{ji}=a_i,\frac{r_i}{a_i}\ge \theta \forall i.$$

$$\text{(}EG)\underset{\mathbf{x}}{\max }\sum _i{a}_i\log r_i-c\sum _{i,j}\log (ϵ+x_{ij})$$

$$\text{subject to}\sum _{j\ne i}{x}_{ji}=a_i,\forall i.$$

$$L(\mathbf{x},\mathbf{\lambda })=\sum _i{a}_i\log r_i-c\sum _{i,j}\log (ϵ+x_{ij})+\sum _i{\lambda }_i(a_i-\sum _{j\ne i}{x}_{ji}).$$

$$\underset{\mathbf{x}}{\max }L(\mathbf{x},\mathbf{\lambda }).$$

$$\log y\le \log y_0+\frac{y-y_0}{y_0}.$$

$$\varphi (\tilde{\mathbf{x}}):=\sum _i{a}_i\log (\sum _{j\ne i}{e}^{{\tilde{x}}_{ij}}),$$

$$\varphi (\tilde{\mathbf{x}})\ge \varphi (\tilde{\mathbf{x}}(t))+(\tilde{\mathbf{x}}-\tilde{\mathbf{x}}(t){)}^T\nabla \varphi (\tilde{\mathbf{x}}(t)).$$

$$L(\tilde{\mathbf{x}},\mathbf{\lambda })\ge g(\tilde{\mathbf{x}}|\tilde{\mathbf{x}}(t)),$$

$$g(\tilde{\mathbf{x}}|\tilde{\mathbf{x}}(t))$$

$$w(x):=\frac{c}{ϵ+x}.$$

$$\tilde{\mathbf{x}}(t+1)=\arg \underset{\tilde{\mathbf{x}}}{\max }g(\tilde{\mathbf{x}}|\tilde{\mathbf{x}}(t)),$$

$${\rho }_i(t):=\frac{r_i(t)}{a_i}$$

$$b_{ij}(\mathbf{x}(t),{\lambda }_i):=\frac{x_{ji}(t)}{{\rho }_j(t)}\frac{1}{{\lambda }_i+w(x_{ji}(t))},$$

$$a_i=\sum _{j\ne i}{b}_{ij}(\mathbf{x}(t),{\lambda }_i),$$

$$x_{ji}(t+1)=b_{ij}(\mathbf{x}(t),{\lambda }_i)$$

$$x_{ij}(t+1)=a_{j}\>x_{ij}(t)\>\frac{a_{i}}{r_{i}(t)}\bigg{/}\sum_{k\neq j}x_{kj}(t)\frac{a_{k}}{r_{k}(t)}.$$

$$D(\mathbf{u},\mathbf{v}):=\sum _i{u}_i\log \frac{u_i}{v_i}-\sum _i(u_i-v_i),\mathbf{u},\mathbf{v}\ge 0.$$

$$\text{(}S)\underset{\mathbf{x}}{\min }D(\mathbf{r},\mathbf{a})+c\sum _{i,j}\log (ϵ+x_{ij})$$

Full text

I Introduction

The unprecedented increase in wireless traffic poses significant challenges for mobile operators, who face extensive infrastructure upgrades to accommodate the rising demand for data. To ease strain on networks, a viable alternative seeks to take advantage of already deployed resources, that presently remain underutilized. For example, in device-to-device communications, devices in close proximity may establish either direct links, or indirect communication via wireless relays, altogether bypassing the cellular infrastructure. In recently launched Wi-Fi internet services, e.g., FON (fon.com), Open Garden (opengarden.com), Karma (yourkarma.com), sharing wireless access is a prominent feature, and subscribers are rewarded for relaying each other’s traffic. In all these scenarios, it is important to design mechanisms which foster cooperation and encourage user contribution, in ways that realize fair and efficient use of pooled resources.

In this paper, we study a network exchange model, where collaborative resource consumption is based on reciprocation. Incentive mechanisms based on reciprocation have been proposed in the context of peer-to-peer and user-provided networks [1], [2]. Participant nodes earn credits (or virtual currency) for assisting other nodes in transmitting their data to the destination. Ideally, reciprocation implies that each peer receives back from the network an amount of resources or utility equal to what he contributed to other users. However, such perfect reciprocation may in general not be feasible, due to constraints arising from network structure/connectivity, and differing resource endowments possessed by nodes, also depending on their position in the network graph. Moreover, peers can typically maintain a limited number of connections (in the popular BitTorrent peer-to-peer protocol users upload to at most four peers). Hence, it seems reasonable to explore situations where graphs representing exchanges of resources among peers are in some sense sparse.

The exchange model considered in this paper builds upon the so-called linear Fisher market in economics (see [3], [4], [2] and references), where each participant aims to receive as much resources as possible from the market. In this model, the optimal resource allocations are captured by a classic convex program discovered by Eisenberg and Gale in 1959 [5]. In the context of peer-to-peer bandwidth trading, the authors in [3] proposed a simple distributed algorithm called proportional–response, that computes solutions to the Eisenberg–Gale program, hence also equilibrium allocations in the resource exchange model: At every time slot, each peer distributes his available resource to other peers in proportion to the resources it received from them in the previous time slot.

Here, we formulate an optimization problem that balances benefits from reciprocation with fixed per-link costs, therefore induces peers to maintain only a few active connections. Clearly, rational peers will not engage in an exchange if costs out-weight potential benefits. We impose sparsity penalties on peer interactions, to reflect the fact that peers often cannot afford the cost of establishing and maintaining a link, the associated communication overhead, etc. fixed costs, or are simply limited by physical constraints, such as limited range of wireless devices. This reciprocity versus sparsity optimization is solved by decentralized tit-for-tat algorithms, whereby peers communicate bids for each other’s resource, so as to approximately compute the sparsest allocations, achieving close-to-perfect reciprocation. Our algorithms implement nonlinear pricing, and extend the proportional–response dynamics of [3] to reinforce interactions where large amounts of resources are exchanged. Starting from a complete graph, the algorithms prescribe how nodes can gradually form a network of exchanges, that progressively gets sparser. As a result, the proposed schemes suggest a network formation model where directed graphs, representing sparsity-constrained resource exchanges, are constructed. The graphs may manifest either direct reciprocation between peers, i.e., both edges $(i,j)$ and $(j,i)$ are typically present in the network graph, or indirect reciprocation, in which case most of the edges do not have their reverse in the graph. We illustrate the formation of resource exchange networks by peers who achieve almost perfect reciprocation with only a small number of connections, and discuss the properties of the sparse graphs in several numerical examples.

From a mathematical standpoint, the sparse exchange algorithms are derived by applying majorization-minimization [6], [7] and reweighted $\ell_{1}$ minimization [8] to a combinatorial problem, and optimize the trade-off between reciprocation and sparsity up to a local optimum. Starting from different initial conditions, different local optima arise, corresponding to different resource allocations and sparse exchange graphs.

II System Model and Background

Consider a network of $N$ peers who exchange resources over a graph $G$ describing connectivity. Exchanges take place only between peers that are neighbours in the graph $G$. Each peer allocates spare resources to other peers, in exchange for their resources (in the future). There exists a single resource/commodity in the network. Peers spend their own spare resource for acquiring resources, i.e., there is no monetary budget. Let $a_{i}$ be the resource endowment of peer $i$. Let $x_{ij}(t)$ be the amount of resource allocated from user $j$ to user $i$ at time $t=0,1,\ldots$. Vector $\mathbf{x}_{i}$ denotes the allocations of peer $i$, and $\mathbf{x}_{-i}$ denotes the allocations of all others. Each peer $i$ allocates the entire budget $a_{i}$ to his neighbours, $a_{i}=\sum_{j\neq i}x_{ji}$, and receives in return a total amount $r_{i}(\mathbf{x}_{-i}):=\sum_{j\neq i}x_{ij}$ of resource. We assume peers value only the resource received from others (not their own spare resource). That is, utility is linear in the amount of received resources $U_{i}(r_{i})=r_{i}$, and each peer $i=1,\ldots,N$, allocates resources to solve

[TABLE]

These $N$ (PEER) problems are intertwined, because each peer’s utility depends on resources received from other peers.

Notation: Subscript in allocation $x_{ij}$ is understood as given from $j$ to $i$, similarly $b_{ij}$ is the bid of peer $j$ for resource of peer $i$, and $\mu_{ij}$ the price peer $j$ charges to $i$.

Motivation. In this paper, we consider an exchange network where the connectivity graph $G$ is complete. Every node can, in principle, engage in exchanges with everybody else. However, we assume that establishing and maintaining exchange links carries a cost, so that peers tend to limit the number of their active connections. This is often the case in practice, where peers choose a few trading partners and avoid spreading themselves thin, so as to reduce overhead, friction etc. costs associated with exchanges. Limits on the number of connections may also arise due to physical or protocol constraints (e.g., in the BitTorrent peer-to-peer protocol). In a slightly different context, reducing transaction costs is a motivation for sparse portfolio selection [9]. Here, in a similar spirit, we use a penalty term that encourages peers to form sparse connections. Our goal is to develop a quantitative model for dynamic formation of exchange networks driven by reciprocation, where the directed graphs representing exchanges are sparse.

**Example. ** The example network of Figure 1 illustrates the graphs implementing sparse exchanges. All four nodes $a,b,c,d$ have resource endowment equal to $1$. Perfect reciprocation can be realized in infinitely many ways, across the $12$ links of the complete graph (left). The sparsest exchange graphs, where each peer gives to exactly one peer $1$ unit of resource, consist of only $4$ links, arranged either in ring (middle) or pairs (right), whence we also see the sparsest solution need not be unique. In addition, the pairs configuration of Figure 1 shows that the sparsest solution may partition the (initially complete) graph into disconnected components.

Previous work. We recap several useful results from a large literature. Exchange network is an instance of a linear Fisher market, for which an equivalent convex formulation was given by Eisenberg and Gale (1959) [5]:

[TABLE]

The objective in (1) resembles the familiar proportionally-fair allocation, where each peer $i$ receives an amount of resources $r_{i}$ proportional to his contribution $a_{i}$. It is also similar to Kelly’s NETWORK problem [10], if the contributions $a_{i}$ are viewed as payments. The receive vector $\mathbf{r}$ achieving optimality in (1) is unique, however the optimal allocation $\mathbf{x}$ is not unique, because the objective in (1) is not strictly concave in $\mathbf{x}$. That is, the same optimal receive vector may be realized with different allocations. The equivalence between (PEER) and (1) can be established as follows. Let $\rho_{j}$ be the price at which peer $j$ “sells” its resource (although no actual monetary payments mediate the exchange). User $i$, by allocating amount $x_{ji}$ to $j$, “purchases” back $x_{ij}=x_{ji}/\rho_{j}$. Hence, the total resource received by user $i$ is $r_{i}=\sum_{j\neq i}x_{ji}/\rho_{j}$. Therefore, to maximize utility in (PEER) user $i$ allocates resources to (and consequently receives resources $x_{ij}>0$ from) only peers with the largest $1/\rho_{j}$, i.e., the cheapest neighbors,

[TABLE]

where ${\cal N}_{i}$ is the set of neighbors of $i$. Now, to find the prices $\boldsymbol{\rho}$, consider the convex program (1), relax the constraints and write the Lagrangian

[TABLE]

The KKT conditions at the saddle point of the Lagrangian imply that either

[TABLE]

or

[TABLE]

From the equations above we deduce that $x_{ij}>0$ if and only if $\rho_{j}=\min_{k\in{\cal N}_{i}}\rho_{k}$, which is precisely condition (2). Therefore, allocations $\mathbf{x}$ and prices $\boldsymbol{\rho}$ solving (PEER) can be computed through the Eisenberg–Gale program (1).

Define the exchange ratio for each peer $i$ as the ratio of the resources $r_{i}$ peer $i$ receives, over the $a_{i}$ he allocates to others. Since $x_{ij}=x_{ji}/\rho_{j}$, by summing over $i$ we get $a_{j}=r_{j}/\rho_{j}$, that is

[TABLE]

Hence, the exchange ratio coincides with the Lagrange multiplier/price $\boldsymbol{\rho}$ in the Eisenberg–Gale program (1). May use the term price and exchange ratio interchangeably.

When network connectivity is given, we summarize the following facts from [2], [3], [4], [11]: Network graph decomposes into components, and resource exchanges take place only within each component. Since the exchange ratio is also the price (Lagrange multiplier) at which each node sells its resource, peers with high exchange ratio are expensive and more constrained, i.e., “poor” and struggle to contribute more resources. At equilibrium, rational peers exchange resource only with their minimum price (cheapest/most generous) neighbours. Exchange can be viewed as a reverse auction: Acting as sellers, peers compete to sell their resource by lowering their prices (raising their bids), whereas, in the role of buyers, they purchase resource from the cheapest (highest “bang-per-buck”) neighbor. Moreover, the prices of peers who exchange resources with each other are inversely proportional. That is, the price at which a node buys resources from peers is equal to the inverse of the price at which he sells his own resource. The upshot is that, by measuring his own price, each peer can infer the price of the peers he exchange resources with. In particular, inexpensive nodes (with prices smaller than one) know they interact with expensive nodes (with price larger than one).

New connections. Previous work typically analysed exchanges in networks where connectivity and resource endowments were a priori fixed and immutable. Then, due to the existing connectivity and neighbouring node endowments, certain nodes may end up receiving significantly less resources than what they contribute to their peers. Unless it is possible to alter either connectivity, or node resource endowments, rational peers with exchange ratio much lower than one have little incentive to engage in exchanges and contribute. Lack of participation will decrease the total amount of resources contributed to the network, i.e., is detrimental to social welfare.

Rational nodes with low prices are motivated to seek out new peers, who are less expensive than the ones they presently interact with. Neighbor selection by a Gibbs sampling algorithm was proposed in [2]. By connecting with richer peers, inexpensive nodes receive more resources, hence their exchange ratio (which is also their price) increases. Along the way, inexpensive peers become more expensive, thereby also less attractive as candidates for resource exchange. In a fully connected network, whenever perfect reciprocity is possible, this balancing act may drive all exchange ratios (prices) to one, i.e. all nodes receive from the network an amount of resources equal to what they contribute – perfect reciprocation.

III Network Formation Problems

Starting from a complete network, we allow peers to gradually form an exchange graph that progressively gets sparser. First, we discuss centralized optimization problems, that aim to identify the sparsest interactions that guarantee a desired level of reciprocation. Then, we introduce variants of the Eisenberg–Gale program (1), where the objective is to balance benefits from equitable allocations with fixed per-link costs, hence form only a few connections. These formulations lead to distributed algorithms that enable peers to compute sparse exchanges in a decentralized manner, by communicating bids for each other’s resource. The algorithms are simple and natural to understand.

III-A Sparse Exchanges with Reciprocation Guarantees

Let $||\mathbf{x}||_{0}$ denote the pseudo-norm that counts the number of nonzero entries in $\mathbf{x}$.

Problem P0. The objective is to find sparsest allocations that achieve a minimum exchange ratio at least $\theta$, where $\theta\leq 1$:

[TABLE]

This is a combinatorial problem (assuming the minimum desired level $\theta$ of reciprocity is feasible), hence intractable. To find an approximate solution, we may replace the nonsmooth $||\mathbf{x}||_{0}$ norm by a smooth proxy. A typical choice, justified by the limit

[TABLE]

is given by the logarithmic approximation $\ell(\mathbf{x}):=\sum_{i,j}\log(\epsilon+x_{i,j})$, leading to the problem

[TABLE]

Hence, the combinatorial objective in (3) has been substituted by a minimization of a concave function (4). This is again hard, and can be tackled as follows.

Problem P1. Successively minimize a linear upper bound to the logarithm in (4), given by

[TABLE]

formed around the previous solution $\mathbf{x}(t)$, for $t=0,1,\ldots$. This amounts to solving a series of linear programs to find a local minimum, hence can be computed efficiently. Moreover, to avoid getting trapped in local minima, a small random perturbation may be employed.

Problem P2. Instead of bounding the logarithm in (4) with a linear upper bound, we bound with a quadratic. More specifically, it holds that $\log(\epsilon+x)\leq q(x,\bar{x})+k,$ where

[TABLE]

for appropriate constant $k$ and small $\delta>0$. Starting from an iterate $\mathbf{x}(t)$, we use (6) to bound (4) and get a quadratic program in $\mathbf{x}$:

[TABLE]

Problem (7) subsequently reduces to a linear system computing $2N$ multipliers from $2N$ linear equations. As in (5), we apply the majorization-minimization procedure [8], hence solve a sequence of quadratic programs to obtain the final solution (IRLS algorithm). Instead of solving each quadratic program completely, we may run one (or a few) iterations towards solution (e.g. fixed-point iteration) of linear system. We anchor a new upper bound to the computed allocations, and resume iterations for the updated linear equations.

The solutions discussed above are centralized. In the following, we focus on distributed algorithms, obtained by balancing reciprocation with a penalty that encourages sparse exchanges.

III-B *Eisenberg–Gale Program with Sparsity Penalty *

We consider an Eisenberg–Gale program (1) augmented with a sparsity promoting term

[TABLE]

Optimization (8) is nonconvex because the objective is a difference of two concave functions. We will derive a distributed algorithm that computes an approximate solution. First, relax the constraints, introduce the multipliers $\boldsymbol{\lambda}$ and write the Lagrangian

[TABLE]

The dual optimization requires solving the relaxed primal

[TABLE]

A local maximum for (9) can be determined in a iterative fashion using majorization-minorization. To that end, we bound the logarithms $\log(\epsilon+x_{ij})$ using

[TABLE]

Next, with a logarithmic change of variables ${\tilde{x}}_{ij}:=\log x_{ij}$, define the function

[TABLE]

so that $\phi(\tilde{\mathbf{x}})=\sum_{i}a_{i}\log r_{i}(\mathbf{x}).$ The convexity of the log-sum-exp function [12, page 74] implies that $\phi$ is convex in $\tilde{\mathbf{x}}$, therefore it holds that

[TABLE]

Making use of inequalities (10) and (11), we lower bound the Lagrangian $L$ by a surrogate function $g$ anchored at $\tilde{\mathbf{x}}(t)$,

[TABLE]

constructed as

[TABLE]

The weights $w$ above are defined as in [8] by

[TABLE]

Function $g(\tilde{\mathbf{x}}|\tilde{\mathbf{x}}(t))$ in (12) is concave in the transformed allocations $\tilde{\mathbf{x}}$ and easy to maximize. Successive maximization of the lower bound (12) yields the iteration

[TABLE]

where, at each time $t=0,1,\ldots$, multipliers $\boldsymbol{\lambda}$ are also updated to satisfy the node endowment constraints in (8). We differentiate (12) with respect to $\tilde{\mathbf{x}}$, transform back to the $\mathbf{x}$ domain, and, after some algebra, arrive at the following solution:

Each peer $i$ communicates at time $t=0,1,\ldots$ its exchange ratio

[TABLE]

to other peers.

Let the bids of peer $j$ for peer $i$’s resource be

[TABLE]

where the weights $w$ are defined in (13) and $\lambda_{i}$ is the multiplier associated with the budget constraint for peer $i$. In market terms, $b_{ij}$ is the amount of resource peer $i$ can purchase from $j$ at price (per unit) $\rho_{j}(t)(\lambda_{i}+w(x_{ji}(t)))$ by paying $x_{ji}(t)$. Note that pricing is nonlinear; price per unit decreases as payment $x_{ji}(t)$ increases, and asymptotically drops to $\lambda_{i}\rho_{j}(t)$ as payment $x_{ji}(t)$ goes to infinity.

Each peer $i$ selects $\lambda_{i}$ to exhaust the entire budget

[TABLE]

this can be computed by bisection search. Check that if endowment $a_{i}$ of peer $i$ is large, then, all other quantities in (16) remaining fixed, multiplier $\lambda_{i}$ (which is also the price associated with $i$’s endowment) will be small, i.e., peer $i$ will be less resource constrained, as expected.

Finally, peer $i$ allocates resources to $j$ proportionally to bids

[TABLE]

where the bids $b_{ij}$ are defined by (15) and multipliers $\lambda_{i}$ solve (16). This is a proportional-response with nonlinear price discrimination. We call the algorithm an EG-sparse proportional-response (EGsPaRse).

When there is no sparsity-promoting penalty $c=0$, the recursion becomes

[TABLE]

Updates (18) coincide with the standard proportional-response dynamics of [3]. To verify this, observe that iteration (18) corresponds to two steps of proportional-response: In the numerator, peer $i$ reciprocates $j$ by charging a constant per-unit price $r_{i}(t)/a_{i}$ (linear pricing), likewise each peer $k$ in the denominator, and for peer $j$ reciprocating $i$ in the entire fraction.

In the general case $c>0$, peers are required to communicate either their exchange ratio, or the multiplier $\lambda$ (which is also related to the exchange ratio). This implicitly assumes peers declare their true ratio. In practice, peers may be unwilling to disclose their exchange ratio (due e.g. to privacy) or strategically misreport it, to extract additional resources. Such strategic/non-cooperative behaviour by peers who anticipate the effect of reporting their ratio may result in loss of optimality.

III-C * An Alternative Formulation: SPaRse Algorithm*

We next turn to an alternative formulation, which leads to an intuitively appealing algorithm. Recall the definition of the Kullback–Leibler divergence between two vectors,

[TABLE]

Inspection of the Eisenberg–Gale program (1) shows it is equivalent to minimizing the divergence $D(\mathbf{a},\mathbf{r})$ between allocated $\mathbf{a}$ and received $\mathbf{r}$ resources (subject to constraints). It is natural to wonder whether we may seek to minimize $D(\mathbf{r},\mathbf{a})$ instead of $D(\mathbf{a},\mathbf{r})$; although divergence is in general not symmetric. It turns out that the former optimization also captures the optimal allocations, a result due to Shmyrev [13] (see also discussion in [4]). The key advantage of this alternative formulation is that it nicely fits the proportional-response dynamics. Hence, we consider a convex program equivalent to (1), obtained from $\min_{\mathbf{x}}D(\mathbf{r},\mathbf{a})$, together with a sparsity penalty:

[TABLE]

Optimization problem (19) is nonconvex; we will derive an algorithm that computes a local minimum using the minorization-majorization procedure [7]. The updates can be expressed in terms of a Bregman divergence $B_{h}$, associated with the convex negative entropy function $h$.

Definition 1

Let $\psi:X\rightarrow\rm I\!R$ be a strongly convex function on a convex set $X$. The Bregman divergence $B_{\psi}:X\times X\rightarrow\rm I\!R$ associated with the strongly convex function $\psi$ is defined by

[TABLE]

The Bregman divergence is a distance–like function, as it satisfies $B_{\psi}(\mathbf{u},\mathbf{v})\geq 0$ for all $\mathbf{u},\mathbf{v}$, thanks to the convexity of $\psi$. For example, $\psi(\mathbf{u})=\frac{1}{2}||\mathbf{u}||^{2}$ induces the usual Euclidean distance $B_{\psi}(\mathbf{u},\mathbf{v})=\frac{1}{2}||\mathbf{u}-\mathbf{v}||^{2}$, however the Bregman divergence is in general not symmetric (for more properties see e.g. [14]). The Bregman distance generated by the negative entropy

[TABLE]

is the Kullback–Leibler divergence:

[TABLE]

This particular choice of Bregman function (instead of usual Euclidean distance) is motivated by the fact that entropy better reflects the geometry of the simplex constraints [14], [4] (so that the latter are easily eliminated).

Let $f(\mathbf{x})$ be the objective function in (19). In the majorization step, a point $\mathbf{x}(t)$ is used to anchor a surrogate function $g(\mathbf{x}|\mathbf{x}(t))$ which upper bounds $f$,

[TABLE]

and is easy to minimize. Function $g$ is chosen to be tight at $\mathbf{x}(t)$, i.e., $f(\mathbf{x}(t))=g(\mathbf{x}(t)|\mathbf{x}(t))$. In the minorization step, the upper bound is minimized with respect to $\mathbf{x}$, generating a sequence

[TABLE]

for each $t=0,1,\ldots.$ We form the surrogate $g(\mathbf{x}|\mathbf{x}(t))$ as follows: Write the divergence $D(\mathbf{r},\mathbf{a})$ as

[TABLE]

Because of (22), negative entropy (21) satisfies

[TABLE]

The divergence $D(\mathbf{r},\mathbf{r}(t))$ in (25) is bounded using Lemma 1 (end of Section III). The logarithm in (19) is bounded by the first-order Taylor expansion

[TABLE]

as is customary in the reweighted $\ell_{1}$ minimization [8] framework. Inserting (24) and (25) in the objective (19) and taking into account inequalities (26) and (32) gives

[TABLE]

The updated allocations $\mathbf{x}(t+1)$ are computed by minimizing the surrogate (27) in (23). After some algebra, also making use of $\partial D(\mathbf{x},\mathbf{x}(t))/\partial x_{ij}=\log x_{ij}-\log x_{ij}(t)$, we get

[TABLE]

Finally, allocations (28) are normalized to satisfy the endowment constraint $\sum_{i\neq j}x_{ij}=a_{j}$ for each peer $j$. We thus arrive at a second algorithm for sparse proportional-response, where nonlinear prices (with an exponential factor) are charged to users:

Each peer $i$ computes the price $\mu_{ji}(t)$ (per unit resource) charged to peer $j\neq i$ at time $t=0,1,\ldots$ as

[TABLE]

Pricing above is nonlinear, because $\mu_{ji}(t)$ depends on the amount of resource $x_{ij}(t)$ (payment) offered to $i$, inside the exponential. The higher the resource $x_{ij}(t)$ (payment) offered by peer $j$ to peer $i$, the lower the price $\mu_{ij}(t)$ (per unit resource) charged to $i$, and the price converges to the exchange ratio $r_{i}(t)/a_{i}$ as payment $x_{ij}(t)$ goes to infinity. Hence, the proposed dynamics reinforce exchanges that involve large amounts of resources.

Next, each peer $i$ communicates to other peers $j$ the price $\mu_{ji}(t)$. Peer $i$ computes the bids of other peers $j$ for $i$’s resource as

[TABLE]

This corresponds to the amount of resource with which $j$ intends to reciprocate $i$. Alternatively, peers can communicate directly the bids instead of prices. Bid $b_{ij}$ is also the number of resource units that peer $i$ can purchase from $j$ with total payment $x_{ji}(t)$, at price $\mu_{ij}(t)$.

Subsequently, peer $i$ allocates his resource to peer $j$ proportionally to the received bids,

[TABLE]

This is a proportional-response with nonlinear price discrimination, where larger amounts of resource are “sold” at lower per-unit price (discount). We call this algorithm a Shmyrev-sparse proportional-response (SPaRse).

Each round of SPaRse has ${\cal O}(N^{2})$ computation and communication complexity. If there is no sparsity penalty (set $c=0$) we recover again recursion (18), which is the standard proportional-response [3]. As will be seen in the numerical results of Section IV, the variant with Equal first round allocation $\mathbf{x}(0)$ tends to generate graphs with mostly direct reciprocation, while Random first round leads to indirect reciprocation, This is likely due to the fact that a random initial allocation adds uncertainty and erases symmetry, so that it gets impossible to recover the more orderly direct reciprocation.

The analysis above can be extended to address a slightly different model, where each peer is constrained by the maximum number of active connections it can maintain at all time slots.

Lemma 1

For all $\mathbf{x},\mathbf{y}\geq 0$ it holds that

[TABLE]

Proof:

Inequality (32) follows from the joint convexity of the function $d(x,y)=x\log(x/y)$ in $(x,y)$ and Jensen’s inequality [4]. ∎

IV Numerical Results

We evaluate the performance of the SPaRse proportional-response algorithm in several numerical examples; EGsPaRse is omitted for brevity. The examples showcase the formation of sparse exchange graphs by peers who communicate bids/prices in a distributed manner, and compute allocations that achieve close to perfect reciprocation (minimum exchange ratio near one). We discuss the influence of the initial split (SPaRse-Equal versus SPaRse-Random variants) on the properties of the induced graphs in terms of direct/indirect reciprocation, the role of the link cost parameter $c$, and the temporal effects (number of iterations $T$) on the sparsity and fairness of the resulting allocations.

The SPaRse algorithm is applied to a $9$-node network, where node endowments are shown in the circles, i.e., node $1$ endowment is $155.81$. After $10,000$ iterations of SPaRse-Equal (with $c=0.1$, $\epsilon=0.01$), a graph with $16$ links is generated, shown in Figure 2, together with the computed allocations $x_{ij}$. The minimum exchange ratio is $0.981$, and the divergence between received and allocated resources is $D(\mathbf{r},\mathbf{a})=0.125$. We see the majority of links are bidirectional: Only links $1\rightarrow 6$, and $6\rightarrow 5$ do not have their reverse in the graph, so among these three nodes indirect reciprocation takes place.

We next consider a $25$-node network, with sample mean node endowment $\bar{a}=106.22$ and standard deviation $std(a)=53.48$, drawn from a lognormal $\log a_{i}\sim{\cal N}(4.5,0.25)$ distribution. We compare influence on the resulting allocations of the Random and Equal initial splits. Figure 3 shows six sample paths of the SPaRse algorithm (with $c=0.2$, $\epsilon=0.01$); solid red line corresponds to Equal initial split of resources, and dashed blue lines correspond to five Random initial splits. It appears that starting with Equal allocation requires more time to converge. We see that, in general, convergence takes place to different allocations, and slightly different minimum exchange ratios, which are larger than $0.9$, not too far from $1$. The top left plot shows that the cardinality of the final allocation is roughly the same under both Random and Equal, i.e., regardless of initial conditions. More interestingly, the bottom left plot in Figure 3 suggests qualitatively different behavior of the two variants: (a) SPaRse-Equal forms a graph that implements direct reciprocation (number of reciprocal links is almost equal to total number of links, at about $45$); while (b) SPaRse-Random generates graphs that implement indirect reciprocation, as there are very few reciprocal links.

The impact of different random initial allocations is quantified in the the same $25$-node network, and node endowments as Figure 3. We run SPaRse-Random $1,000$ times (with $c=0.1$, $\epsilon=0.01$), each time with a different random split of resources in the first round. Runs are $5,000$ iterations long, by then allocations have converged. We record four performance metrics: (i) the cardinality of the final allocation (the number of directional links in the resulting exchange network), (ii) the reciprocity (i.e., the number of links for which their reciprocal is also in the graph), (iii) the minimum exchange ratio over the $25$ nodes, and (iv) the divergence $D(\mathbf{r},\mathbf{a})$ between received $\mathbf{r}$ and allocated $\mathbf{a}$ resources. Figure 4 shows histograms and mean values (vertical black line) for all $4$ metrics. We see that SPaRse-Random usually achieves a minimum exchange ratio larger than $0.92$, with sparse graphs consisting of less than $50$ edges, out of $25\times 24=600$ totally in a complete graph with $25$ nodes. The top histograms (cardinality of $\mathbf{x}$, reciprocity) once more indicate that graphs generated by SPaRse-Random manifest mostly indirect reciprocation, since (on the average) only about $4$ out of the $46$ links are reciprocal.

The role of sparsity parameter $c$ is examined in Figure 5. In a $N=11$ node network, five endowment vectors are randomly drawn from the same lognormal distribution as before. For each endowment vector we run SPaRse-Equal with different sparsity parameters $c$ and record the cardinality of the resulting allocation, and the divergence $D(\mathbf{r},\mathbf{a})$, to get five cardinality and divergence curves. The duration of each run is $10^{4}$ iterations. As $c$ decreases, the algorithm computes more fair allocations (smaller divergence, larger minimum exchange ratio), but takes longer to converge. Decreasing $c$ below $0.05$ (while average endowment is about $100$) yields close to zero divergence, i.e., perfect reciprocation (Figure 5, right). However, for $c$ smaller than $0.05$, and when computations stop after $10^{4}$ iterations, we see that almost zero divergence is accompanied by an increase in number of links in the graph (Figure 5, left).

The discussion above suggests that, by tuning the parameters $c$ and $\epsilon$, our model can generate graphs with various levels of sparsity and reciprocation, which also evolve temporally as the allocation of resources changes over the course of time. Apart from the static graphs that arise after SPaRse converges, one may also take a snapshot of the network at some finite time, during the transient. For example, at time $t=0$ let us start with the $9$-node endowments of Figure 2 and apply SPaRse in a complete graph, which sparsifies as time elapses. By stopping early after $500$ iterations, we obtain the graph shown in Figure 6. This consists of $35$ links (as compared to $16$ links in Figure 2), where only link $3\rightarrow 1$ is not directly reciprocated (but has small allocation $0.02$). The minimum exchange ratio is $0.998$, and divergence $D(\mathbf{r},\mathbf{a})=0.001$, while the respective values in Figure 2 were $0.981$ and $0.125$. Therefore, graph in Figure 6 is less sparse than Figure 2, but realizes more fair exchanges. A common feature of the allocations in both Figures 2 and 6 is that low endowment nodes apparently never exchange resources with each other.

V Conclusion

We studied a resource exchange network where exchanges among nodes are based on reciprocity. To incorporate costs of establishing and maintaining active connections, we imposed sparsity penalties on peer interactions. Finding the sparsest graphs that achieve a certain level of reciprocation is in general NP-hard. We proposed decentralized algorithms, that enable peers to approximately compute the sparsest allocations, by generalized proportional-response dynamics, with nonlinear pricing. Numerical results illustrate the performance of the SPaRse algorithms and the formation of exchange graphs by peers who achieve close-to-perfect reciprocation (minimum exchange ratio near one), in a network with a limited number of active connections.