Reconciliation k-median: Clustering with Non-Polarized Representatives

TL;DR

This paper introduces a new clustering variant called reconciliation k-median that balances minimizing assignment costs with reducing polarization among cluster representatives, with theoretical guarantees and practical benefits.

Contribution

It formulates a novel k-median variant incorporating disagreement penalties, providing approximation guarantees and demonstrating effectiveness on real-world datasets.

Findings

The algorithm achieves constant-factor approximation guarantees.

Clustering results show reduced polarization without significant loss of fidelity.

Experimental validation on Twitter, Congress data, and news sources supports practical utility.

Abstract

We propose a new variant of the k-median problem, where the objective function models not only the cost of assigning data points to cluster representatives, but also a penalty term for disagreement among the representatives. We motivate this novel problem by applications where we are interested in clustering data while avoiding selecting representatives that are too far from each other. For example, we may want to summarize a set of news sources, but avoid selecting ideologically-extreme articles in order to reduce polarization. To solve the proposed k-median formulation we adopt the local-search algorithm of Arya et al. We show that the algorithm provides a provable approximation guarantee, which becomes constant under an assumption on the minimum number of points for each cluster. We experimentally evaluate our problem formulation and proposed algorithm on datasets inspired by the motivating applications. In particular, we experiment with data extracted from Twitter, the US Congress voting records, and popular news sources. The results show that our objective can lead to choosing less polarized groups of representatives without significant loss in representation fidelity.

Tables

Table 1. Table 1. Summary of the datasets.

Name	Number of	Number of	Distance functions
	clients	facilities
Twitter	$\mathrm{3\hspace{0.17em}302\hspace{0.17em}362}$	$500$	Shortest path, Spectral embedding $+$ Euclidean
Congress	$420$	$420$	Euclidean
Domains	$\mathrm{6\hspace{0.17em}104}$	$469$	Weighted Jaccard $+$ Mentions, Latent space $+$ Euclidean

Table 2. Table 2. Top-16 news sources for different values of λ 𝜆 \lambda , using latent representations.

$\lambda =0$
Domain	Frequency	Ideology	Mentions
nydailynews.com	1.000	-0.114	10 191
politico.com	1.000	-0.073	36 184
slate.com	1.000	-0.341	14 364
cbsnews.com	0.687	-0.057	4 394
buzzfeed.com	0.687	-0.262	11 683
twitchy.com	0.537	0.469	13 192
westernjournalism.com	0.462	0.450	3 562
9news.com	0.462	-0.016	349
politifact.com	0.350	-0.240	3 097
cbsloc.al	0.337	-0.081	2 526
christianpost.com	0.337	0.337	383
theatlantic.com	0.312	-0.176	6 883
newrepublic.com	0.312	-0.335	1 626
lifenews.com	0.200	0.483	3 657
6abc.com	0.200	-0.252	819
usatoday.com	0.112	-0.064	18 513

Table 3. Table 3. Number of iterations on Twitter dataset (avg/max).

Metric	$k=2$	$k=4$	$k=8$
Shortest path	$2.5/4$	$2.8/5$	$3.09/7$
Spectral, $\gamma =2$	$2.18/3$	-	-
Spectral, $\gamma =4$	-	$2.73/5$	-
Spectral, $\gamma =8$	-	-	$3.13/6$
Spectral, $\gamma =10$	$2.32/4$	$2.8/6$	$3.32/7$
Spectral, $\gamma =100$	$2.45/4$	$2.75/5$	$2.58/4$

Equations

$$cost(S)=\sum _{c\in C}\underset{s\in S}{\min }\{d(c,s)\}+\frac{\lambda }{2}\sum _{s_i\in S}\sum _{s_j\in S}d(s_i,s_j).$$

$$cost(S)=\sum _{s\in S}\sum _{c\in N_S(s)}d(c,s)+\frac{\lambda }{2}\sum _{s_i\in S}\sum _{s_j\in S}d(s_i,s_j).$$

$$\min$$

$$\mathbf{x}\in \{0,1{\}}^n\text{mbox}and{\mathbf{x}}^{T}1=k.$$

$$\lambda_{i}\geq\mu_{i}\geq\lambda_{n-m+i}~{}~{}~{}\mbox{for $i=1,\dots,m$}$$

$$k{\lambda }_1(\mathbf{D})\ge \sum _{s_i\in S}\sum _{s_j\in S}d(s_i,s_j)\ge \sum _{i=n-k+1}^n{\lambda }_i^2(\tilde{}\mathbf{D}).$$

$$cost(S)\le 2(\lambda k+5)cost(O).$$

$$cost(S)\le \max \{11,4\lambda \}cost(O).$$

$$d(f,g)=1-\frac{{\sum }_{c\in S_f\cap S_g}\log n_{cf}}{W}.$$

$$\underset{\mathbf{x}}{\min }$$

$$\mathbf{x}\in \{0,1{\}}^n\text{mbox}and{\mathbf{x}}^{T}1=k.$$

$${\tilde{A}}_{ij}=\{\begin{array}{ll}{A}_{ij}& (i,j)\in E\\ 0& i=j\\ 2& \text{mbox}otherwise.\end{array}$$

$${\tilde{A}}_{ij}\le {\tilde{A}}_{i,\ell }+{\tilde{A}}_{\ell ,j}.$$

$$\mathbf{x}=\underset{\begin{array}{c}\mathbf{x}\in \{0,1{\}}^n\\ {\mathbf{x}}^{T}1=k\end{array}}{\mathrm{argmin}}{\mathbf{x}}^T\tilde{A}\mathbf{x},$$

$$\mathbf{x}=\underset{\begin{array}{c}\mathbf{x}\in \{0,1{\}}^n\\ {\mathbf{x}}^{T}1=k\end{array}}{\mathrm{argmax}}{\mathbf{x}}^{T}A\mathbf{x}.$$

$${\mathbf{x}}^T\tilde{A}\mathbf{x}=\sum _i\sum _j{\tilde{A}}_{ij}\mathbb{I}\{x_i=x_j=1\}=M+2N,$$

$$M=\left|\left\{(i,j)\mid x_i=x_j=1\right\}\cap E\right|,$$

$$N=\left|\left\{(i,j)\mid x_i=x_j=1\right\}\cap \bar{E}\right|.$$

$${\mathbf{x}}^{T}A\mathbf{x}=\sum _i\sum _j{A}_{ij}\mathbb{I}\{x_i=x_j=1\}=M.$$

$${\mathbf{x}}^T\tilde{A}\mathbf{x}=-{\mathbf{x}}^{T}A\mathbf{x}+2(k^2-k).$$

$$5f(O)-f(S)\ge \sum _{(o_i,s_j)\in \Sigma }\left(f(S-s_j+o_i)-f(S)\right).$$

$$f(S-s_j+o_i)+\lambda g(S-s_j+o_i)\ge f(S)+\lambda g(S),$$

$$g(S-s_j+o_i)-$$

$$=\frac{1}{2}\sum _{x\in S-s_j+o_i}\sum _{y\in S-s_j+o_i}d(x,y)-\frac{1}{2}\sum _{x\in S}\sum _{y\in S}d(x,y)$$

$$=\sum _{\begin{array}{c}s\in S\\ s\ne s_j\end{array}}d(o_i,s)-\sum _{s\in S}d(s,s_j).$$

$$\sum _{(o_i,s_j)\in \Sigma }(g(S$$

$$=\sum _{(o_i,s_j)\in \Sigma }\left(\sum _{\begin{array}{c}s\in S\\ s\ne s_j\end{array}}d(o_i,s)-\sum _{s\in S}d(s,s_j)\right).$$

$$\sum _{(o_i,s_j)\in \Sigma }\left(f(S-s_j+o_i)-f(S)\right)$$

$$+\lambda \sum _{(o_i,s_j)\in \Sigma }\sum _{\begin{array}{c}s\in S\\ s\ne s_j\end{array}}d(o_i,s)-\lambda \sum _{(o_i,s_j)\in \Sigma }\sum _{s\in S}d(s_j,s)\ge 0,$$

$$5f(O)+\lambda \sum _{(o_i,s_j)\in \Sigma }\sum _{\begin{array}{c}s\in S\\ s\ne s_j\end{array}}d(o_i,s)\ge f(S)+\lambda \sum _{(o_i,s_j)\in \Sigma }\sum _{s\in S}d(s_j,s).$$

$$\sum _{(o_i,s_j)\in \Sigma }\sum _{\begin{array}{c}s\in S\\ s\ne s_j\end{array}}d(o_i,s).$$

$$\sum _{(o_i,s_j)\in \Sigma }\sum _{\begin{array}{c}s\in S\\ s\ne s_j\end{array}}d(o_i,s)\le k(g(O)+f(O)).$$

$$\sum _{(o_i,s_j)\in \Sigma }\sum _{s\in S}d(s_j,s).$$

$$2\sum _{(o_i,s_j)\in \Sigma }\sum _{s\in S}d(s_j,s)\ge g(S).$$

Full text

To solve the proposed $k$-median formulation we adopt the local-search algorithm of Arya et al. (Arya et al., 2004), We show that the algorithm provides a provable approximation guarantee, which becomes constant under an assumption on the number of points for each cluster. We experimentally evaluate our problem formulation and proposed algorithm on datasets inspired by the motivating applications. In particular, we experiment with data extracted from Twitter, the US Congress voting records, and popular news sources. The results show that our objective can lead to choosing less polarized groups of representatives without significant loss in representation fidelity.

Clustering; $k$-median; polarization; committee selection; data mining; approximation algorithms.

††journalyear: 2019††copyright: iw3c2w3††conference: Proceedings of the 2019 World Wide Web Conference; May 13–17, 2019; San Francisco, CA, USA††booktitle: Proceedings of the 2019 World Wide Web Conference (WWW’19), May 13–17, 2019, San Francisco, CA, USA††doi: 10.1145/3308558.3313475††isbn: 978-1-4503-6674-8/19/05††ccs: Theory of computation Facility location and clustering††ccs: Theory of computation Discrete optimization††ccs: Applied computing Sociology

1. Introduction

Consider the problem of summarizing a set of news articles on a given topic. A standard approach to this problem is clustering: design a distance function that captures similarity between the news articles and apply a clustering algorithm on the resulting metric space. Common clustering formulations, such as $k$-median or $k$-means, can be used (Jain and Dubes, 1988). The original set of input news articles can then be summarized by the (small) set of cluster representatives. In some cases, however, we may be interested in selecting cluster representatives that are not too far from each other. For example, we may want to find a set of representative news articles that are not too extreme so that they can provide a basis for constructive deliberation. This motivation is similar to recent proposals in the literature that aim to reduce polarization in social media (Garimella et al., 2017a) and balance the users’ content consumption (Garimella et al., 2017b). In this work we are interested in developing computational methods for clustering data in a way that the disagreement of the cluster representatives is minimized.

Another motivating example appears in the context of electing a $k$-member committee to represent a set of individuals, such as the employees of an organization or the members of a political party. Assuming that all individuals have public opinions on a set of issues, clustering these individuals on an opinion-representation space will give a committee that faithfully represents the set of individuals with respect to the issues under consideration. Despite providing a good representation, however, a committee elected with a standard clustering approach may fail to reach consensus due to potential heterogeneity within the committee members. Heterogeneity within elected members of an assembly is a widely acknowledged problem in politics — for instance, division of representatives often results in paralysis in various left-wing political formations.111https://www.theguardian.com/commentisfree/2019/feb/19/podemos-spanish-politics As in the previous example, we are interested in electing a committee in a way that the disagreement of the elected members is minimized while ensuring a faithful representation of the constituents.

Motivated by the previous examples we introduce a new formulation of the $k$-median problem, where in addition to the $k$-median objective we also seek to minimize disagreement between the cluster representatives. As it is customary, we consider a metric space $(X,d)$, where $d$ is a distance function for objects in $X$. We distinguish two subsets of $X$, the set of facilities $F$ and the set of clients $C$. The goal is to select a set of $k$ facilities $S\subseteq F$ — the cluster representatives — so as to minimize the overall cost

[TABLE]

The first term is the same as in the standard $k$-median, and expresses the cost of serving each client by its closest selected facility. The second term is the one introduced in this paper and expresses disagreement between cluster representatives. The parameter $\lambda$ determines the relative importance of the two terms. Despite clustering being one of the most well-studied problems in statistical data analysis, and the numerous formulations and problem variants that have been studied in the literature, to our knowledge, the problem defined above has not been considered before.

As expected, the problem defined by optimizing Equation (1) is $\mathbf{NP}$-hard; in fact optimizing each of the two terms separately is an $\mathbf{NP}$-hard problem. Given the hardness of the problem, it is compelling to consider algorithms with provable approximation guarantees. For the $k$-median algorithm several approximation algorithms exist (Charikar et al., 1999; Li and Svensson, 2016). A local-search algorithm, which is simple to implement and scalable to large datasets, has been proposed by Arya et al. (Arya et al., 2004). The algorithm starts with an arbitrary solution and considers a swap of $p$ selected facilities with $p$ non-selected facilities; the swap is materialized if the cost improves, and the process continues until the cost cannot be improved. Arya et al. show that this algorithm achieves an approximation guarantee equal to $3+2/p$, and the running time is $\mathcal{O}(n^{p})$. In particular, for $p=1$, the algorithm gives an approximation guarantee equal to $5$, while the running time is linear.

In this paper we show how to adapt the local-search algorithm of Arya et al. (Arya et al., 2004) for the problem we consider and obtain an approximation guarantee $\mathcal{O}(k)$ in the case $F=C$. The proposed algorithm considers 1-facility swaps, i.e., $p=1$. Furthermore, when the clusters of the obtained solution have equal size of $\Omega(\lambda k)$, the approximation factor becomes $11$, i.e., a constant. We complete the analysis of the proposed problem by deriving bounds on the objective function.

Our contributions in this paper are summarized as follows.

• •

We introduce the reconciliation $k$-median problem, a novel clustering problem formulation where we aim to optimize the data representation cost plus a term for agreement between the cluster representatives.

• •

We adapt the local-search algorithm of Arya et al. (Arya et al., 2004) and obtain provable approximation guarantees for the proposed clustering problem.

• •

We run experiments on datasets extracted from the Twitter social network, US Congress voting records, and popular news sources. The results show that the proposed objective can lead to the choice of less polarized groups of representatives, as measured by a well-known method for ideology estimation (Barberá, 2014) and an objective estimate of the political leaning of news sources.

Erratum: A previous version of this paper incorrectly omitted some of the necessary assumptions. In particular, the statement of Theorem 1 overlooked the requirement that the client and the facility sets be the same. On the other hand, an argument in the proof of Theorem 2 required the cardinality of the clusters to be similar (and them being the same is sufficient). These details have been corrected in the present version. The proof of Theorem 1, which was omitted in the conference publication due to space constraints, is included here as well.

The rest of the paper is structured as follows. In Section 2 we present a brief overview to the literature that is most related to our work. In Section 3 we formally define the reconciliation $k$-median problem. In Section 4 we present the local-search algorithm and state its approximability properties. In Section 5 we present our experimental evaluation, while Section 6 is a short conclusion. To improve readability, the hardness proof of the reconciliation $k$-median problem and the proof of the approximation guarantee of the local-search algorithm are presented in the Appendix.

2. Related work

Data clustering is one of the most well-studied problems in data analysis, with applications in a wide range of areas (Jain and Dubes, 1988). Among the numerous formulations that have been proposed, in this paper we focus on the $k$-median problem setting, which has been studied extensively in the theoretical computer-science literature. Charikar et al. (Charikar et al., 1999) gave the first constant-factor approximation algorithm for the $k$-median problem, followed by improvements that relied on both combinatorial and LP-based algorithms (Charikar and Guha, 1999; Jain et al., 2002; Jain and Vazirani, 2001). In this paper we build upon the local-search algorithm of Arya et al. (Arya et al., 2004). This is a simple-to-implement and scalable algorithm that had been offering the best performance guarantee for over a decade. The current best approximation guarantee is $2.67+\epsilon$, provided by the algorithm of Byrka et al. (Byrka et al., 2014b), which optimizes a part of the algorithm of Li and Svensson (Li and Svensson, 2016). However, the algorithm is not practical.

Variants of the $k$-median problem have also been considered, including the Euclidean $k$-median (Ahmadian et al., 2017), capacitated $k$-median (Byrka et al., 2014a), ordered $k$-median (Byrka et al., 2018), and more. To our knowledge, however, this is the first work to study the problem of $k$-median clustering with a penalty on the disagreement of the cluster representatives. Instead, researchers have studied the problem of selecting $k$ points to maximize the sum of pairwise distances, i.e., the dispersion of the selected point set. Several constant-factor approximation algorithms have been proposed for the maximum-dispersion problem (Fekete and Meijer, 2004; Hassin et al., 1997). However, the maximization makes the problem different and it is not clear how to adapt those algorithms in our setting.

One of our motivating applications is summarization of social-media content with the aim of reducing polarization and balancing the information content delivered to users. This is a relatively new research area that is receiving increasing interest (Barberá, 2014; Garimella et al., 2017a, b; Munson et al., 2013; Musco et al., 2017). However, to the best of our knowledge, none of the proposed approaches uses a clustering formulation.

The second motivating application is election of committees and representatives. In some cases, election questions can also be formulated as voting problems. Voting in general has been studied extensively in social sciences and economics literature. From the algorithmic perspective, researchers have studied questions about voting in social networks and concepts such as liquid and viscous democracy (Boldi et al., 2009; Yamakawa et al., 2007). In addition to being not directly related to our paper, this line of work does not directly model agreement between elected representatives.

3. Problem formulation

3.1. Preliminaries

We formulate our problem in the general setting of metric facility location (Jain and Vazirani, 2001). We consider a metric space $(X,d)$ and two subsets $F,C\subseteq X$, not necessarily disjoint. The set $F$ represents facilities, and the set $C$ represents clients. A special case of interest is when clients and facilities are defined over the same set, i.e., $F=C$. In our discussion we consider the more general case that the sets of clients and facilities are disjoint. The function $d:X\times X\rightarrow\mathbb{R}$ is a distance measure between pairs of points in $X$. When $c\in C$ and $s\in F$, the distance $d(c,s)$ represents the cost of serving client $c$ with facility $s$. The number of facilities is denoted by $m=|F|$ and the number of clients by $n=|C|$.

The goal is to open $k$ facilities — i.e., choose $k$ points in $F$ — such that the cost of serving each client in $C$ with the nearest selected facility is minimized. Given a set of facilities $S\subseteq F$, with $|S|=k$, and $s\in S$, we use $N_{S}(s)$ to denote the set of clients served by facility $s$ in the solution $S$, that is, $N_{S}(s)=\{c\in C\mid s=\operatorname*{\arg\!\min}_{x\in S}d(c,x)\}$. In the facility location formulation each facility has an associated cost, which is incurred if the facility is opened (selected). The objective is to minimize the total cost of serving all clients plus the cost of opened facilities, while there is no restriction on the number of opened facilities. When the cost of opening each facility is zero and it is required to open at most $k$ facilities, the problem is known as $k$-median.

3.2. Reconciling cluster representatives

The problems described above, facility location and $k$-median, are commonly used to find cluster representatives without regard to the relative position of the representatives themselves. As discussed in the introduction, our goal is to modify the problem definition so as to find solutions in which the cluster representatives are close to each other. To achieve our goal we propose the following clustering variant, which we name reconciliation $k$-median.

Problem 1 (recon-$k$-median).

Given a metric space $(X,d)$, two sets $F,C\subseteq X$, $k\in\mathbb{N}$, and a real number $\lambda>0$, find a set $S\subseteq F$ with $|S|=k$, so as to minimize the cost function

[TABLE]

In order to characterize the hardness of this problem, we analyze the two terms of the objective in isolation. The first term, which results from setting $\lambda=0$, is equivalent to the classical metric $k$-median problem, shown to be $\mathbf{NP}$-hard by Papadimitriou (Papadimitriou, 1981). To analyze the second term, we define the following equivalent problem, which asks to find a subset of $k$ points that minimize the sum of pairwise distances in a metric space, i.e., minimum pairwise distances (mpd).

Problem 2 (mpd).

Given a metric space $(X,d)$, a subset of objects in the metric space $F=\{x_{1},\dots,x_{n}\}\subseteq X$, and a number $k\in\mathbb{N}$, with $k<n$, define a matrix $\mathbf{A}\in\mathbb{R}^{n\times n}$ as $\mathbf{A}_{ij}=d(x_{i},x_{j})$ for all $x_{i},x_{j}\in F$. The goal is to find a binary vector $\mathbf{x}$ of dimension $n$ that has exactly $k$ coordinates equal to 1 and minimizes the form $\mathbf{x}^{T}\mathbf{A}\,\mathbf{x}$. In other words, we want to find

[TABLE]

The following lemma establishes that there exists no polynomial-time algorithm to find the exact solution to the mpd problem, unless $\mathbf{P}=\mathbf{NP}$. The proof is given in the Appendix.

Lemma 0.

Problem mpd is $\mathbf{NP}$-hard.

Lemma 1 establishes that optimizing separately the second term of the objective function (2) is an $\mathbf{NP}$-hard problem. Note, however, that the hardness of the two terms in Problem 1 does not immediately imply the hardness of the overall problem. Consider, for instance, an objective of the form $\min_{\mathbf{x}}\left\{f(\mathbf{x})+(c-f(\mathbf{x}))\right\}$. Even though optimizing $f$ can be an arbitrary $\mathbf{NP}$-hard problem, the overall problem has a constant value, and thus, it is trivial to optimize — there is nothing to be done. We now show that Problem 1 is indeed $\mathbf{NP}$-hard.

Lemma 0.

Problem recon-$k$-median is $\mathbf{NP}$-hard.

Proof.

Consider an instance of the mpd problem, for a given set $F$ and a number $k$, and form an instance $(F,C,k,\lambda)$ of the recon-$k$-median problem, where $C$ is any arbitrary set of clients with $F\cap C=\emptyset$. Set $d(s,c)=\frac{1}{2}\max_{s_{i},s_{j}\in F}\{d(s_{i},s_{j})\}$, for all $c\in C$ and $s\in F$. Note that the distance function $d$ is still a valid metric. We have that $\sum_{c\in C}\min_{s\in S}d(c,s)$ is constant for any potential solution set $S\subseteq F$, which implies that optimizing recon-$k$-median is equivalent to optimizing mpd. Thus, from Lemma 1 we obtain that recon-$k$-median is $\mathbf{NP}$-hard. ∎

3.3. Bounds on the objective

To complete the analysis of the recon-$k$-median problem we offer bounds on the objective, which can be used to evaluate the quality of the solution obtained by any algorithm for the problem at a given instance. For this to be useful, the bounds need to be non-trivial and as close as possible to the optimal solution. We will first show how to obtain a lower bound on the second term of the objective, that is, $\sum_{s_{i},s_{j}\in S}d(s_{i},s_{j})$. We first introduce the following definition.

Definition 0.

Given two sequences of real numbers $\lambda_{1}\geq\lambda_{2}\geq\dots\geq\lambda_{n}$ and $\mu_{1}\geq\mu_{2}\geq\dots\geq\mu_{m}$, with $m<n$, we say that the second sequence interlaces the first if

[TABLE]

We will employ the following result from Haemers (Haemers, 1995).

Lemma 0.

Let $\mathbf{A}$ be a symmetric $n\times n$ matrix, and let $\mathbf{B}$ be a principal submatrix of $\mathbf{A}$. Then the eigenvalues of matrix $\mathbf{B}$ interlace those of matrix $\mathbf{A}$.

We now state the following result.

Theorem 5.

Let $\mathbf{D}$ be the pairwise distance matrix of facilities of an instance of recon-$k$-median problem. Define matrix $\widetilde{}\mathbf{D}$ so that $\widetilde{}\mathbf{D}_{ij}=\sqrt{\mathbf{D}_{ij}}$, that is, a matrix whose entries are the square roots of the entries of $\mathbf{D}$. Let $\lambda_{i}(\widetilde{}\mathbf{D})$ denote the $i$-th absolutely largest eigenvalue of $\widetilde{}\mathbf{D}$. Then

[TABLE]

Proof.

Recall that for any real symmetric matrix $\mathbf{D}$, $\|\mathbf{D}\|_{F}^{2}=\sum_{i}\sigma_{i}^{2}(\mathbf{D})=\sum_{i}\lambda_{i}^{2}(\mathbf{D})$. It is easily seen that if $\mathbf{x}$ is a binary vector with exactly $k$ entries equal to 1, $\mathbf{x}^{T}\mathbf{D}\,\mathbf{x}$ is equal to the sum of the entries of a $k\times k$ principal submatrix of $\mathbf{D}$, which is in turn equal to the squared Frobenius norm of the corresponding submatrix of $\widetilde{}\mathbf{D}$. Combined with Lemma 4, this proves the lower bound.

The upper bound follows immediately from the variational characterization of the eigenvalues (Kreyszig, 1978, chap. 9.2). ∎

A lower and upper bound on the first term of the objective can simply be given by $\sum_{c\in C}\min_{s\in F}d(c,s)$ and $\sum_{c\in C}\max_{s\in F}d(c,s)$, respectively. This is useful only for problem instances where the number of facilities is relatively small compared to the number of clients.

4. The local-search algorithm

In this section we present the proposed algorithm for the recon-$k$-median problem and state its properties. The algorithm uses the local-search strategy, proposed by Arya et al. (Arya et al., 2004), but adapted for the objective function of recon-$k$-median. The algorithm starts with an arbitrary solution consisting of $k$ selected facilities. It then proceeds in iterations. In each iteration it considers whether it is possible to swap a selected facility with a non-selected facility and obtain an improvement in the objective score. If such an improvement is possible, the corresponding swap is performed. The algorithm terminates when no such swap is possible. At each point during its execution, the algorithm maintains a set of $k$ clusters over the set of clients, and a selected facility for each cluster, defined by assigning each client to its closest selected facility. Pseudocode of this local-search procedure is given in Algorithm 1.

For the analysis we denote by $S=\{s_{1},\ldots,s_{k}\}\subseteq F$ the solution returned by LocalSearch, and $O=\{o_{1},\ldots,o_{k}\}\subseteq F$ an optimal solution. As mentioned before we use the notation $N_{S}(s)$ to denote the set of clients that are assigned to facility $s$ in the solution $S$, and $N_{O}(o)$ to denote the set of clients that are assigned to facility $o$ in the optimal solution $O$.

To analyze the performance of LocalSearch we follow the ideas of Arya et al. (Arya et al., 2004). The proofs are included in the Appendix.

As a result, in the case $F=C$, the LocalSearch algorithm yields a $\mathcal{O}(\lambda k)$-factor approximation guarantee on the quality of the solution achieved.

Theorem 1.

Let $(C,C,k,\lambda)$ be an instance of the recon-$k$-median problem. Let $S$ be a solution returned by the LocalSearch algorithm, and let $O$ be an optimal solution. Then

[TABLE]

Furthermore, we are able to improve the analysis and obtain an approximation guarantee that does not depend on the number of facilities $k$. For the improved result we need to make the mild assumption that the number of clients in any cluster of the optimal solution and the solution returned by the algorithm is $\Omega(\lambda k)$. In particular, we have.

Theorem 2.

Let $(C,C,k,\lambda)$ be an instance of the recon-$k$-median problem. Let $S$ be a solution returned by the LocalSearch algorithm, and let $O$ be an optimal solution. Assume that $|N_{S}(s)|\geq\lceil 2\lambda\rceil k$, $|N_{O}(o)|\geq\lceil 2\lambda\rceil k$ and $|N_{S}(s)|=|N_{O}(o)|$ for all $s\in S$, $o\in O$. Then

[TABLE]

The running time of the LocalSearch algorithm is $\mathcal{O}(nmk)$ per iteration. For most applications $k$ is considered to be a constant. When the number of facilities $m$ is of the same order of magnitude with the number of clients $n$, e.g., in the important special case $F=C$, the algorithm has quadratic complexity per iteration. However, in many applications the number of facilities is significantly smaller than the number of clients. Thus, we expect that the algorithm is very efficient in practice for those cases.

We also note that Arya et al. (Arya et al., 2004) show how to perform swaps of $p$ facilities simultaneously and obtain an improved performance guarantee at the expense of increased running time. In our case, the penalty term in our objective, which captures the disagreement among the cluster representatives, makes the analysis significantly more complex and it is not clear how to use simultaneous swaps in order to achieve a similar quality-performance trade-off. Thus, this direction is left for future work.

5. Experimental evaluation

We perform experiments to assess the proposed concept of clustering with reconciliation of representatives, as well as the performance of the proposed LocalSearch algorithm.222Our implementation of the algorithm is available at https://github.com/brunez/recon-kmedian-ls Our objective is to evaluate by some objective measure whether the proposed problem formulation, as well as natural variations, can lead to a choice of representatives or sources that are more moderate, less polarizing and more likely to reach consensus.

To enrich the experimental setting and produce a more interesting set of empirical observations, we relax some of the requisites of our theoretical results. Namely, observe that in order to prove the approximation guarantees of the LocalSearch algorithm, we require that the distance function satisfy the properties of a metric, and that it be the same for facilities and clients. However, we believe that in practical scenarios, one might benefit from considering a wider set of options, especially if we consider the exploratory nature of clustering algorithms. Therefore, we measure dissimilarity between objects using functions that do not necessarily qualify as metrics, and we consider different ones for facilities and clients.

At a high level, our experimental methodology is as follows: We start with a dataset for which clients and facilities model a natural clustering problem and for which a distance function $d$ is available. For the facilities of the dataset we seek to obtain a polarity score $\pi$, which is independent of the distance function $d$: facilities with similar polarity scores $\pi$ are more likely to agree. In addition, facilities with scores closer to the middle of the spectrum are less likely to disseminate extremist ideologies. We then apply our clustering algorithm with varying values of the parameters $k$ and $\lambda$. We are interested in answering the following questions:

• Q1.

How does the agreement between selected representatives or the polarization of information sources (measured by the independent polarity score $\pi$) change as a function of $\lambda$? In other words, can we get more reconciled representatives or less extreme sources by increasing the weight of the disagreement penalty term (second term of the objective function (2))?

• Q2.

How does the $k$-median score change as a function of $\lambda$? In other words, can we find solutions with more reconciled representatives but without significant loss in representation fidelity (i.e., first term of the objective function (2))?

• Q3.

What is the impact of the parameter $k$ on both polarity score and $k$-median score?

For our experimental evaluation we use the following datasets.333The datasets are available at https://doi.org/10.5281/zenodo.2573954

Twitter: The dataset, obtained by Lahoti et al. (Lahoti et al., 2018), consists of a set of politically active Twitter accounts. We remove stubs — i.e., accounts that follow only one account and have no followers — resulting in $3\,302\,362$ accounts. Out of those we consider 500 popular ones — with at least $50\,000$ followers — as candidate facilities, that is, representatives. As remarked in the beginning of section 5, we can extend the proposed framework by considering different metrics for the two terms of the objective function. This corresponds to a practical setting where the agreeability of the selected representatives is measured differently than their affinity to their respective consituents. Specifically, for this dataset we consider the following distance functions.

1. (1)

Facility-Client: We compute distances between facilities and clients as the length of the shortest path between two Twitter accounts in the undirected follower Twitter graph.444The follower graph corresponds to a snapshot taken in July 2017. 2. (2)

Facility-Facility: To compute distances between facilities we use shortest-path distances, as before. We also use Euclidean distances on the spectral embedding with ${\gamma}$ components, as described by Belkin et al. (Belkin and Niyogi, 2002). We scale the resulting distance matrix so that the average of all entries is equal to that of the shortest-path distance matrix. This way we ensure that the magnitude of $\lambda$ has an equivalent effect using the different metrics.

Congress: We collect roll call voting records from the present US Congress using the public domain congress API.555https://github.com/unitedstates/congress/wiki We build a dataset where each row corresponds to a Congress representative and each column is a binary variable representing the issue being voted. Missing values are imputed using class-conditional means, where the classes we consider are the two parties: democrats and republicans. “Present” and “Not voting” votes are considered to be missing. We omit votes where all representatives are missing. We also omit representatives for whom we could not obtain an ideology estimate using the approach described below, or who missed too many votes. For this dataset, we use the Euclidean distance between the vectors corresponding to each of the representatives. To make this experiment closer to a plausible practical scenario, we restrict half of the facilities to be democrats and the other half to be republicans. In addition, clients are served by the closest facility of the same party.

Domains: We combined the domain-related data described in the work of Bakshy et al. (Bakshy et al., 2015) with the Twitter dataset. The set of facilities consisted of 469 domains hosting the news sources most often shared on the Facebook social network. The client set is comprised of 6 104 of the most politically active Twitter users. We consider two alternatives for computing the distances.

1. (1)

Mentions: Given a facility $f$ and a client $c$, let $n_{cf}$ be the number of times a tweet by $c$ contains a link to $f$. Then $d(f,c)=(n_{cf}+1)^{-1}$. To compute the pairwise distances between facilities we do the following. Consider two facilities, $f$ and $g$. Let $S_{f}$ (respectively $S_{g}$) be the set of clients that have tweeted a link to $f$ (respectively $g$) at least once. We define $W=\sum_{c\in S_{f}\cup S_{g}}\left(\log n_{cf}\mathbb{I}\{c\in S_{f}\}+\log n_{cg}\mathbb{I}\{c\notin S_{f}\}\right)$, where $\mathbb{I}$ is the indicator function. Then

[TABLE]

We define $\log 0=0$. Since the objective function of recon-$k$-median (Equation (1)) sums over all ordered pairs of facilities, this distance function is in effect symmetric when applied to our problem. Note that this is akin to the Jaccard index for set similarity, but each element is weighted with a measure of its relevance. 2. (2)

Latent: We construct a matrix $A$ such that $A_{ij}$ is the number of times a tweet by user $j$ contains a link to domain $i$. We compute the singular value decomposition $A=U\Sigma V^{T}$ and extract the latent representation for both domains and users in the first 9 components (which account for 50% of the total Frobenius norm of $A$). If $k=9$, domain $i$ is represented as $U_{i,:k}$ and user $j$ as $V_{i,:k}$. To compute both facility-facility and facility-client distances we take the Euclidean distances between the corresponding latent representations.

The characteristics of the datasets are summarized in Table 1.

For all datasets, in order to compute the objective of recon-$k$-median we take averages instead of the sums of distances. Note that this amounts to scaling both sums, so it is equivalent to setting $\lambda$ to a particular value. The advantage of taking averages is that $\lambda$ has an impact at small values, i.e., at “small” factors of 1.

Ground truth polarity scores ($\pi$). To measure the polarity scores of the facilities we employ different methods depending on the dataset.

In the case of Twitter and Congress we use the approach described by Barberá (Barberá, 2014), which estimates the ideological leaning of a Twitter account as a real value. For Twitter, we use polarity scores collected at the same time as the follower graph (July 2017). For Congress, we collected the ideological estimates in May 2018. Using this method and the proposed datasets, all the elicited polarity scores are between -3 and 3. We measure the polarity of the chosen representatives as follows. Given a solution $S=\{s_{1},\dots,s_{k}\}$, let $\pi(s_{i})$ denote the estimated polarity of facility $s_{i}$. We define the polarity of solution $S$ as the sample standard deviation of the set $\{\pi(s_{1}),\dots,\pi(s_{k})\}$.

For Domains, we use the ideological leaning score associated to each domain as described in the work of Lahoti et al. (Lahoti et al., 2018). These scores were computed roughly as the fraction of interactions (visits or shares) by conservative users, out of total interactions. We translated the scores so that they fall between -0.5 (left) and 0.5 (right). In this scenario, we are interested in choosing less polarized sources. We therefore measure the polarity of the chosen set as the $\ell$-2 norm of the vector $(\pi(s_{1}),\dots,\pi(s_{k}))$.

Results on Twitter dataset. We run the algorithm on the Twitter dataset setting the number of selected facilities to $k=2,4,8$ and $\lambda=0,2^{i}/10$ for $i=1,\dots,6$. For computing the pairwise distances between facilities we use either the shortest path metric or the spectral embedding with $\gamma=k,10,100$. Figure 1 illustrates the results. We depict, as a function of $\lambda$, the polarity of the chosen representatives, measured as described above, along with the $k$-median cost of the solution — i.e., the first term of the objective function (2). We run the algorithm 40 times for each setting and report the average and standard deviation bands. We can see that increasing the value of $\lambda$ leads to significantly less polarized representatives in various cases. The effect is most noticeable for larger values of $k$, and particularly consistent using the spectral embedding with 100 components. An interesting result on this dataset is that we can achieve significant decreases in polarity without incurring much additional $k$-median cost. That is, it is possible to elect a much more agreeable committee — with respect to the chosen polarity measure — without notable loss in representation fidelity.

Results on Congress dataset. We run the algorithm on the Congress dataset, using the same configurations as for Twitter. Figure 2 illustrates the results. Here, the decrease in polarity is only clear in the case $k=2$, for values of $\lambda$ at least 0.8. It should be noted, however, that the voting data and the polarity scores come from completely different sources. It would therefore be interesting to carry out further experiments with these data.

Results on Domains dataset. We run the algorithm on the Domains dataset, using the same configurations as for Twitter but considering $k=16$ as well, as in the case of news sources it is practical to consider larger sets. Figure 3 shows the results, using the distance function defined in Equation (4) (w-jaccard) and the latent representation (latent). We run each configuration 80 times and report average results and standard deviation bands. The reduction in polarity is noticeable, in particular using the latent representation with larger values of $k$. For very small sets of news sources (e.g., $k=2$) the method does not exhibit a reduction in polarity. In order to gain further insight on the impact of the penalty term, we report an example of the news sources that appear more frequently as $\lambda$ increases. Specifically, we take a case where decrease in polarity is noticeable (latent, $k=8$, $\lambda=0.8$). We then collect the 16 sources that appear the most, and do the same for $\lambda=0$. The results are shown in Table 2. For each domain, we report the frequency (i.e., the fraction of times it was part of the solution out of the 80 runs), the ideological score and the number of times it was mentioned in the collected tweets.

Number of iterations. Even though the time complexity of the local-search algorithm per iteration is not too high, a legitimate concern to be raised is the possibility that it might require a large number of iterations to converge. Our observations, however, suggest that in practice a small number of iterations — where by iteration we understand the inspection of all candidate changes — are necessary. Table 3 shows the average and maximum number of iterations for the Twitter dataset.

6. Conclusions

We have considered the problem clustering data so as to optimize the total representation cost plus an additive term to penalize disagreement among the chosen representatives. The proposed problem, which we name reconciliation $k$-median, has applications in summarizing data with non-polarized representatives, as well as in electing $k$-member committees that are more likely to reach consensus. We have shown the proposed problem to be $\mathbf{NP}$-hard and derived bounds on the objective. Inspired by the literature on related problems, we have analyzed a local-search algorithm in this context and derived approximation guarantees, of factor $\mathcal{O}(\lambda k)$ in the general setting, and constant under mild assumptions. Through experiments on real data coming from a social network and voting records, we have shown empirically how the proposed formulation can lead to the choice of less polarized groups of representatives, as measured by a well-known method for ideology estimation, as well as less ideologically-extreme sets of news sources. This work opens various enticing directions for future inquiry. First it would be interesting to determine whether the approximation guarantees can be improved, as well as to attempt to find tight examples to know the possible extent of said improvement. Second, it would be interesting to perform further experiments on similar and other datasets. It is particularly compelling to improve our understanding of how different metrics can interact with known methods for estimating polarization.

Acknowledgements. This work was supported by three Academy of Finland projects (286211, 313927, and 317085), and the EC H2020 RIA project “SoBigData” (654024).

Appendix

6.1. Hardness results

Here we prove Lemma 1, which is a key ingredient of the proof of hardness for recon-$k$-median. Before we proceed, we provide a definition of Densest $k$-subgraph, a well-known $\mathbf{NP}$-hard optimization problem which we employ in our reduction.

Problem 3.

*(densest $k$-subgraph — d$k$s)

Given a simple graph $G=(V,E)$ with $|V|=n$ and adjacency matrix $A$, and a number $k\in\mathbb{N}$ with $k<n$, find*

[TABLE]

We can now prove the aforementioned lemma.

Proof of lemma 1.

We proceed by reduction from Densest $k$-subgraph (d$k$s).

Consider an instance of d$k$s, $G=(V,E)$, $|V|=n$ with adjacency matrix $A$. We define a matrix $\tilde{A}$ as follows:

[TABLE]

Notice that this matrix is symmetric, the diagonal (and nothing else) is zero, and since $\min_{i\neq j}\tilde{A}_{ij}=1$ and $\max_{i\neq j}\tilde{A}_{ij}=2$, for all $i$, $j$, and $\ell$

[TABLE]

We want to show that if

[TABLE]

then

[TABLE]

We can write

[TABLE]

where we have defined

[TABLE]

That is, $M$ is the number of pairs in $\mathbf{x}$ with a corresponding edge in $G$, and $N$ is the number of pairs in $\mathbf{x}$ without a corresponding edge in $G$. Similarly,

[TABLE]

Furthermore, note that $M+N=2{k\choose 2}=k^{2}-k$.

It follows that for all vectors $\mathbf{x}\in\{0,1\}^{n}$, with $\mathbf{x}^{T}\mathbf{1}=k$, it is

[TABLE]

Since $2(k^{2}-k)$ is a constant, it follows that a vector $\mathbf{x}\in\{0,1\}^{n}$, with $\mathbf{x}^{T}\mathbf{1}=k$, minimizes $\mathbf{x}^{T}\tilde{A}\,\mathbf{x}$ (the mpd objective) if and only if it maximizes $\mathbf{x}^{T}{A}\,\mathbf{x}$ (the d$k$s objective).

∎

6.2. Approximation guarantees

Given a set of clients $C$, a set of facilities $F$, and any subset of facilities $S\subseteq F$, we employ the following notation:

• –

$f(S)=\sum_{x\in C}d(x,s(x))$, where $s(x)$ is the facility in $S$ assigned to client $x\in C$;

• –

$g(S)=\frac{1}{2}\sum_{x\in S}\sum_{y\in S}d(x,y)$;

• –

$N_{S}(s)$ is the set of clients served by facility $s$ in $S$.

We rely on the following crucial facts. First, a result from the work of Arya et al. (Arya et al., 2004).

Lemma 0 ((Arya et al., 2004)).

Let $O$ be an optimal set of $k$ facilities and $S$ an arbitrary set of $k$ facilities. There exists a set of $k$ pairs $\Sigma=\{(o_{i},s_{j})\mid i,j\in[k]\}\subseteq O\times S$ satisfying the following properties:

1. (1)

Every $o\in O$ is considered in exactly one pair. 2. (2)

Every $s\in S$ is considered in at most two pairs. 3. (3)

We can choose the set of pairs $\Sigma$ such that the following inequality holds:

[TABLE]

Second, by definition of local optimality, for a locally optimal solution $S=\{s_{1},\dots,s_{k}\}$ and an optimal solution $O=\{o_{1},\dots,o_{k}\}$, we have

[TABLE]

for any $i,j\in[k]$.

We now have the ingredients for the proof of Theorem 1, from Section 4.

Proof of theorem 1.

Since $S$ and $S-s_{j}+o_{i}$ differ in one facility only, we have

[TABLE]

We consider a set of pairs $\Sigma$ satisfying the properties of lemma 1. Summing the above difference over all $k$ pairs we get

[TABLE]

Therefore, summing inequality (6) over all pairs in $\Sigma$ we get

[TABLE]

and using inequality (5) and rearranging we obtain

[TABLE]

We consider this inequality, and modify it so that it becomes dependent only on factors of $f(O)$, $g(O)$, $f(S)$, $g(S)$. To accomplish this, we will consider the set of pairs $\Sigma$.

First, let us examine the following quantity:

[TABLE]

For each term $d(o_{i},s),s\in S,(o_{i},s_{j})\in\Sigma$, we consider three cases:

• •

$s\in O$. Then $d(o_{i},s)\leq g(O)$.

• •

$s\notin O$ and $s\in N_{O}(o_{i})$. Then $d(o_{i},s)\leq f(O)$.

• •

$s\notin O$ and $s\notin N_{O}(o_{i})$. Then $d(o_{i},s)\leq d(o_{i},o_{j})+d(o_{j},s)$, where $s\in N_{O}(o_{j})$, and thus $d(o_{i},s)\leq f(O)+g(O)$.

Now, consider the sum $\sum_{(o_{i},s_{j})\in\Sigma}\sum_{\begin{subarray}{c}s\in S\\ s\neq s_{j}\end{subarray}}d(o_{i},s)$ and observe that

• •

for every $i$, $o_{i}$ is in $k-1$ terms of the sum;

• •

for every $j$, $s_{j}$ is in at most $k$ terms of the sum.

Thus,

[TABLE]

We now examine the quantity

[TABLE]

In particular, we will show that

[TABLE]

We consider the set $P\subseteq S$ of facilities in some swap, i.e., P=\{$$s\in S such that $(o,s)\in\Sigma$ for some o\in O$$\}. For any facility $s\in P$, note that all the entries from $g(S)$ that involve terms $d(s,s^{\prime})$ are present in expression (9), and so we can trivially bound them.

Thus, we can assume that $k$ is even and $|P|=k/2$, that is, all facilities in $S$ either are in two swaps from $\Sigma$, or in no swaps at all. We number the facilities in $P$ $1$ through $k/2$, and the rest $k/2+1$ through $k$.

We bound each term of $g(S)$ corresponding to facilities $i,j$ not in $P$, $i<j$ as

[TABLE]

Note we can bound all such terms this way, with each term in the upper bound appearing at most twice, and being part of the sum in expression (9). Thus, we get inequality (10).

Now, consider inequality (7), and apply inequalities (8) and (10). We obtain

[TABLE]

∎

Proof of theorem 2.

We consider inequality (7), and modify it so that it becomes dependent only on factors of $f(O)$, $g(O)$, $f(S)$, $g(S)$. To accomplish this, we will consider the set of pairs $\Sigma$.

First, let us examine the following quantity:

[TABLE]

By the triangle inequality, we have that the sum corresponding to each pair $(o_{i},s_{j})\in\Sigma$ is bounded as follows:

[TABLE]

where $o_{h_{s}}$ is such that $N_{O}(o_{h_{s}})\cap N_{S}(s)\neq\emptyset$ and $x\in N_{O}(o_{h_{s}})\cap N_{S}(s)$. If $N_{S}(s)\subseteq N_{O}(o_{i})$, we consider $d(o_{i},s)\leq d(o_{i},o_{h_{s}})+d(o_{i},x)+d(x,s)$, choosing $o_{h_{s}}$ as described below.

For notational convenience, we define

[TABLE]

for each $(o_{i},s_{j})\in\Sigma$ — note that each $o_{i}$ appears exactly once in the pairs in the set $\Sigma$, so the pair $(o_{i},s_{j})$ is uniquely determined by $o_{i}$.

We want to choose the entries of inequality (12) such that

• $-$

every term of the form $d(o_{i},o_{h_{s}})$ appears at most twice;

• $-$

every term of the form $d(o_{h_{s}},x)$ appears once;

• $-$

every term of the form $d(x,s)$ appears once.

To achieve this, we need a set of replacements such that for each $\sigma(o_{i})$, the $k-1$ corresponding replacements contain $k-1$ distinct entries of the forms $d(o_{i},o_{h_{s}})$, $d(o_{h_{s}},x)$, $d(x,s)$.

We define a function $\mu$ that maps each $x\in C$ to a pair $(o_{i},s_{j})\in O\times S$ such that $x\in N_{S}(s_{j})\cap N_{O}(o_{i})$. Since $|N_{S}(s_{i})|\geq k$, $|N_{O}(o_{i})|\geq k$ and $|N_{S}(s)|=|N_{O}(o)|$ for all $s\in S,o\in O$, we can choose a subset $\tilde{C}\subseteq C$ of $k^{2}$ points such that each $s\in S$, as well as each $o\in O$, appears in $k$ of the $k^{2}$ pairs associated to the chosen points.

We now argue that said choice is indeed possible. Consider a bipartite multigraph, where the vertices of each partition correspond to the facilities in $S$ and $O$, respectively. We add an edge between $s$ and $o$ for each client served by the two facilities in the corresponding solution. That we can pick these $k^{2}$ clients satisfactorily follows from the fact that every regular bipartite multigraph is 1-factorable (Akiyama and Kano, 2011).

We can now conclude that

• $-$

every element of the form $d(o_{i},o_{j})$ can only appear either in $\sigma(o_{i})$ or in $\sigma(o_{j})$. Since for all $\sigma(o_{i})$, the entry $d(o_{i},o_{j})$ is unique, each $d(o_{i},o_{j})$ appears at most twice;

• $-$

for any element of the form $d(o,x),o\in O$, $x$ is unique;

• $-$

for any element of the form $d(s,x),s\in S$, $x$ is unique.

We have established the following inequality

[TABLE]

Considering each cluster in both $S$ and $O$ to have at least $\lceil 2\lambda\rceil k$ points, we can extend the result to account for $\lambda$. To do this, we repeat the above choice of clients $\lceil 2\lambda\rceil$ times and obtain

[TABLE]

To see this, note that for every entry of the form $d(o,x)$ or $d(s,x)$, $x$ can still be chosen to be unique.

Now, consider inequality (7) again. Multiplying it by $2$ we get

[TABLE]

By inequalities (14) and (10) we obtain the following:

[TABLE]

Therefore,

[TABLE]