Factorization Bandits for Online Influence Maximization

TL;DR

This paper introduces a novel online influence maximization method leveraging network assortativity and factorization of influence probabilities, significantly improving learning efficiency and effectiveness in social networks.

Contribution

It proposes a factorization-based online learning algorithm utilizing network assortativity, which is a new approach in influence maximization.

Findings

Significant regret reduction demonstrated in experiments.

Effective estimation of influence probabilities through latent factorization.

Empirical results on real-world networks validate the approach.

Abstract

We study the problem of online influence maximization in social networks. In this problem, a learner aims to identify the set of "best influencers" in a network by interacting with it, i.e., repeatedly selecting seed nodes and observing activation feedback in the network. We capitalize on an important property of the influence maximization problem named network assortativity, which is ignored by most existing works in online influence maximization. To realize network assortativity, we factorize the activation probability on the edges into latent factors on the corresponding nodes, including influence factor on the giving nodes and susceptibility factor on the receiving nodes. We propose an upper confidence bound based online learning solution to estimate the latent factors, and therefore the activation probabilities. Considerable regret reduction is achieved by our factorization based online influence maximization algorithm. And extensive empirical evaluations on two real-world networks showed the effectiveness of our proposed solution.

Tables

Table 1. Table 1 . Robustness to dimensionality misspecification.

	IMFB-1	IMFB-5	IMFB-20	IMFB-60	$ϵ$-greedy	CUCB	IMLinUCB-Node2Vec	IMLinUCB	DILinUCB
NetHEPT	4657.80	4711.30	4731.90	4715.80	2882.10	3022.30	4262.40	4392.50	2368.80
Flickr	979.80	1044.88	1054.56	1044.17	706.14	880.37	744.52	800.86	715.89

Table 2. Table 2 . Reward comparison under different environment settings on Flickr dataset.

	IMFB	$ϵ$-greedy	CUCB	IMLinUCB-Node2Vec	IMLinUCB	DILinUCB
$K=50$	203.92	161.80	194.71	130.47	179.24	131.31
$K=100$	405.50	295.09	352.51	243.68	332.28	257.73
$K=300$	1054.56	706.14	880.37	744.52	800.86	715.89
$K=1,000$	2737.86	2224.10	2508.25	2013.49	2157.98	2198.64
$\eta =0$	1054.56	706.14	880.37	744.52	800.86	715.89
$\eta \sim U(-0.05,0.05)$	1455.83	1120.56	1241.26	1205.89	1267.67	1131.39
$\eta \sim U(-0.1,0.1)$	1956.80	1641.30	1740.00	1856.83	1921.10	1711.80
$\eta \sim U(-0.3,0.3)$	2766.07	2587.10	2690.30	2659.34	2736.78	2590.18
$c=0.5$	432.45	359.19	367.75	383.46	412.45	350.28
$c=1.0$	1054.56	706.14	880.37	744.52	800.86	715.89
$c=1.5$	2727.07	1863.94	2407.70	2035.44	2479.88	2278.18
$c=2.0$	3884.68	3517.8	3776.86	3598.31	3658.57	3526.53

Equations

$$p_e={\mathbf{\theta }}_{g_e}^{\mathsf{T}}{\mathbf{\beta }}_{r_e},$$

$$\underset{\{{\mathbf{\theta }}_v,{\mathbf{\beta }}_v{\}}_{v\in \mathcal{V}}}{\min }\sum _{t=1}^T\sum _{e\in {\tilde{\mathcal{E}}}_t}({\mathbf{\theta }}_{g_e}^{\mathsf{T}}{\mathbf{\beta }}_{r_e}-y_{e,t}{)}^2+{\lambda }_1\sum _{v\in \mathcal{V}}\Vert {\mathbf{\theta }}_v{\Vert }_2+{\lambda }_2\sum _{v\in \mathcal{V}}\Vert {\mathbf{\beta }}_v{\Vert }_2$$

$${\mathbf{A}}_{v,t+1}$$

$${\mathbf{b}}_{v,t+1}$$

$${\mathbf{C}}_{v,t+1}$$

$${\mathbf{d}}_{v,t+1}$$

$$\Vert {\hat{\mathbf{\theta }}}_{v,t}-{\mathbf{\theta }}_v^{\ast }{\Vert }_{{\mathbf{A}}_{v,t}}$$

$$|p_e^{\ast }-{\hat{p}}_{e,t}|$$

$$\displaystyle R^{\alpha\gamma}(T)\leq O\big{(}\frac{dB}{\alpha\gamma}$$

$$\displaystyle+\sqrt{T|\mathcal{V}^{\text{receiving}}||\mathcal{V}|D^{\text{in}}}\log(TD^{\text{in}}))\big{)}$$

$$\mathbb{E}[R_t^{\alpha \gamma }]\le \frac{1}{\alpha \gamma }\mathbb{E}[f_{{\mathbf{P}}^{\ast }}({\mathcal{S}}^{\ast })-f_{{\mathbf{P}}^{\ast }}({\mathcal{S}}_t)]$$

$$\le B\sum _{e\in {\tilde{\mathcal{E}}}_t}|{\bar{p}}_{e,t}-p_e^{\ast }|$$

$$\mathbb{E}[R_t^{\alpha \gamma }]$$

$$\le \frac{1}{\alpha \gamma }\mathbb{E}[f_{{\mathbf{P}}^{\ast }}({\mathcal{S}}^{\ast })-f_{{\mathbf{P}}^{\ast }}({\mathcal{S}}_t)]$$

$${\xi }_{t-1}=\{$$

$$+{\alpha }_{r_e}^{\theta }\Vert {\hat{\mathbf{\theta }}}_{v_{r_e},i-1}{\Vert }_{{\mathbf{C}}_{r_e,i-1}^{-1}}+2q^{2i},\forall e\in \mathcal{E},\forall i\le t\}$$

$$\mathbb{E}[R_t^{\alpha \gamma }]$$

$$\le \frac{\mathbb{P}({\xi }_{t-1})}{\alpha \gamma }\mathbb{E}[f_{{\bar{\mathbf{P}}}_t}({\mathcal{S}}^{\ast })-f_{{\mathbf{P}}^{\ast }}({\mathcal{S}}_t)|{\xi }_{t-1}]+\mathbb{P}({\bar{\xi }}_{t-1})(L-K)$$

$$\le \frac{\mathbb{P}({\xi }_{t-1})}{\alpha \gamma }\mathbb{E}[f_{{\bar{\mathbf{P}}}_t}({\mathcal{S}}_t)-f_{{\mathbf{P}}^{\ast }}({\mathcal{S}}_t)|{\xi }_{t-1}]+\mathbb{P}({\bar{\xi }}_{t-1})(L-K)$$

$$f_{{\bar{\mathbf{P}}}_t}({\mathbf{S}}_t)-f_{{\mathbf{P}}^{\ast }}({\mathbf{S}}_t)\le B\sum _{e\in {\tilde{\mathcal{E}}}_t}|{\bar{p}}_{e,t}-p_e^{\ast }]|$$

$${\text{R}}^{\alpha \gamma }(T)$$

$$\le \frac{B}{\alpha \gamma }\sum _{t=1}^T\sum _{e\in {\tilde{\mathcal{E}}}_t}|{\bar{p}}_{e,t}-p_e^{\ast }|+\mathbb{P}({\bar{\xi }}_{t-1})(L-K)$$

$$\displaystyle\textbf{R}^{\alpha\gamma}(T)\leq O\big{(}\frac{dB}{\alpha\gamma}($$

$$\displaystyle+\sqrt{T|\mathcal{V}^{\text{receiving}}||\mathcal{V}|D^{\text{in}}\log(TD^{\text{in}})})\big{)}$$

$$\sum _{t=1}^T\sum _{e\in {\tilde{\mathcal{E}}}_t}|{\bar{p}}_{e,t}-p_e^{\ast }|$$

$$\le O($$

$$+d\sqrt{T|{\mathcal{V}}^{\text{receiving}}||\mathcal{V}|D^{\text{in}}\log (T{D}^{\text{in}})})$$

$$\sum _{t=1}^T\sum _{e\in {\tilde{\mathcal{E}}}_t}|{\bar{p}}_{e,t}-p_e^{\ast }|$$

$$\le \sum _{t=1}^T\sum _{e\in {\tilde{\mathcal{E}}}_t}({\alpha }_{v_{g_e}}^{\beta }\Vert {\hat{\mathbf{\beta }}}_{g_e,i-1}{\Vert }_{{\mathbf{A}}_{g_e,i-1}^{-1}}+{\alpha }_{r_e}^{\theta }\Vert {\hat{\mathbf{\theta }}}_{v_{r_e},i-1}{\Vert }_{{\mathbf{C}}_{v_{r_e},i-1}^{-1}}+2q^t)$$

$$=\sum _{t=1}^T\sum _{u\in {\mathcal{V}}_{{\mathcal{S}}_t,}^{\text{giving}}}{\alpha }_u^{\beta }\Vert {\hat{\mathbf{\beta }}}_{u,t}{\Vert }_{{\mathbf{A}}_{u,t}^{-1}}+\sum _{t=1}^T\sum _{v\in {\mathcal{V}}_{{\mathcal{S}}_t,}^{\text{receiving}}}{\alpha }_v^{\theta }\Vert {\hat{\mathbf{\theta }}}_{v,i-1}{\Vert }_{{\mathbf{C}}_{v,i-1}^{-1}}+\sum _{t=1}^T|{\tilde{\mathcal{E}}}_t|2q^t)$$

$$\le {\alpha }^{\beta }\sum _{t=1}^T\sum _{u\in {\mathcal{V}}_{{\mathcal{S}}_t,}^{\text{giving}}}\Vert {\hat{\mathbf{\beta }}}_{u,t}{\Vert }_{{\mathbf{A}}_{u,t}^{-1}}+{\alpha }^{\theta }\sum _{t=1}^T\sum _{v\in {\mathcal{V}}_{{\mathcal{S}}_t,}^{\text{receiving}}}\Vert {\hat{\mathbf{\theta }}}_{v,i-1}{\Vert }_{{\mathbf{C}}_{v,i-1}^{-1}}+\sum _{t=1}^T|{\tilde{\mathcal{E}}}_t|2q^t)$$

$$\sum _{t=1}^T\sum _{u\in {\mathcal{V}}_{{\mathcal{S}}_t,}^{\text{giving}}}\Vert {\hat{\mathbf{\beta }}}_{u,t}{\Vert }_{{\mathbf{A}}_{u,t}^{-1}}$$

$$\le \sqrt{T|{\mathcal{V}}^{\text{giving}}||\mathcal{V}|d\frac{D^{\text{out}}\log (1+\frac{T{D}^{\text{out}}}{d{\delta }^2})}{\log (1+1/{\delta }^2)}}$$

$$\sum _{t=1}^T\sum _{v\in {\mathcal{V}}_{{\mathcal{S}}_t,}^{\text{receiving}}}\Vert {\hat{\mathbf{\theta }}}_{v,t}{\Vert }_{{\mathbf{C}}_{v,t}^{-1}}$$

$$\le \sqrt{T|{\mathcal{V}}^{\text{receiving}}||\mathcal{V}|d\frac{D^{\text{in}}\log (1+\frac{T{D}^{\text{in}}}{d{\delta }^2})}{\log (1+1/{\delta }^2)}}$$

$$\sum _{t=1}^T\sum _{e\in {\mathcal{E}}_{{\mathcal{S}}_t}}|{\bar{p}}_{e,t}-p_e^{\ast }|$$

$$\le {\alpha }^{\mathbf{\beta }}\sqrt{T|{\mathcal{V}}^{\text{giving}}||\mathcal{V}|d\frac{D^{\text{out}}\log (1+\frac{T{D}^{\text{out}}}{d{\delta }^2})}{\log (1+1/{\delta }^2)}}$$

$$+{\alpha }^{\mathbf{\theta }}\sqrt{T|{\mathcal{V}}^{\text{receiving}}||\mathcal{V}|d\frac{D^{\text{out}}\log (1+\frac{T{D}^{\text{in}}}{d{\delta }^2})}{\log (1+1/{\delta }^2)}}+|{\mathcal{E}}_O|\frac{2q(1-q^T)}{1-q}$$

$$\le O($$

$$+d\sqrt{T|{\mathcal{V}}^{\text{receiving}}||\mathcal{V}|D^{\text{in}}\log (T{D}^{\text{in}})})$$

$$\sum _{t=1}^T\sum _{u\in {\mathcal{V}}_{{\mathcal{S}}_t,}^{\text{giving}}}\Vert {\hat{\mathbf{\beta }}}_{u,t}{\Vert }_{{\mathbf{A}}_{u,t}^{-1}}^2\le |\mathcal{V}|\frac{d{D}^{\text{out}}\log (1+\frac{T{D}^{\text{out}}}{d{\delta }^2})}{\log (1+1/{\delta }^2)}$$

$$\sum _{t=1}^T\sum _{v\in {\mathcal{V}}_{{\mathcal{S}}_t,}^{\text{receiving}}}\Vert {\hat{\mathbf{\theta }}}_{v,t}{\Vert }_{{\mathbf{C}}_{v,t}^{-1}}^2\le |\mathcal{V}|\frac{d{D}^{\text{in}}\log (1+\frac{T{D}^{\text{in}}}{d{\delta }^2})}{\log (1+1/{\delta }^2)}$$

Full text

Online influence maximization; Factorization bandit; Network assortativity; Regret analysis

††copyright: none††journalyear: 2019††copyright: acmcopyright††conference: The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 4–8, 2019; Anchorage, AK, USA††price: 15.00††doi: 10.1145/3292500.3330874††isbn: 978-1-4503-6201-6/19/08††ccs: Information systems Social advertising††ccs: Theory of computation Online learning algorithms††ccs: Theory of computation Social networks

1. Introduction

Online social networks play a vital role in the spread of information, ideas, and influence among people in their modern life (Mislove et al., 2007; McPherson et al., 2001). They have been actively utilized as a dissemination and marketing platform. For instance, in viral marketing, a marketer tries to select a set of customers with great influence for a new product promotion. With a fixed budget on the number of selections, a marketer aims to maximize the spread of this influence, which is referred to as the influence maximization problem (Kempe et al., 2003; Kitsak et al., 2010; Centola and Macy, 2007). In typical solutions for influence maximization, a social network is modeled as a graph with nodes representing users and edges, associated with the activation probability, representing the connections or relationship between users. Influence is propagated through the network under a specific diffusion model, such as independent cascade model and linear threshold model (Kempe et al., 2003).

Most existing influence maximization solutions assume the activation probability is known beforehand. However, in many real-world social networks, this information is not observable. Solutions have been proposed to estimate the activation probabilities from a set of cascades which consist of logged actions by the network users in the past (Goyal et al., 2010; Netrapalli and Sanghavi, 2012; Saito et al., 2008; Bourigault et al., 2016). However, as the observations are independently collected from the learning algorithms, bias might be introduced by the logging mechanism (Morgan and Winship, 2014; El-Sayed et al., 2012). To combat with biases in offline influence estimation, Aral and Walker (Aral and Dhillon, 2018) proposed randomized online experiments for offline data collection. But it is usually very expensive to carry out such experiments. This motivates the studies of online influence maximization (Chen et al., 2013, 2016; Vaswani et al., 2015; Wen et al., 2017; Vaswani et al., 2017), in which seed nodes are purposely selected by a learning agent to improve its quality in influence estimation and influencer selection on the fly. The foundation of this line of solutions is the combinatorial bandits (Chen et al., 2013), in which a set of arms are pulled together at each round, and the outcome is only revealed as a whole over the set of pulled arms. Mapping it back to the influence maximization problem, each node in the network is considered as an arm, and at each round the received reward on the selected set of seed nodes is the number of their activated nodes.

Most of existing online influence maximization solutions (Chen et al., 2013, 2016) model the activation probability on the edges independently, which unfortunately cannot capture how social influence forms in real networks (Newman, 2002; Aral and Walker, 2012). To clarify, here we are referring to the independence/dependence in the estimation of influence across network edges instead of how influence diffuses. First, this independence assumption prevents the model from realizing the dependency among the influence pattern on related nodes, i.e., the so-called assortative mixing (Newman, 2002). For example, in “influentials hypothesis” (Watts and Dodds, 2007) the tendency that a user is likely to influence his/her neighbors is defined as the influence of this node and the tendency that he/she is likely to be influenced is defined as the node’s susceptibility. In Aral and Walker’s work (Aral and Walker, 2012), via randomized experiments, they found that a node’s influence should be separated from its susceptibility: influential individuals are less susceptible to be influenced than non-influential individuals. This directly suggests the activation probability on an edge should be modeled by both its end nodes’ influence and susceptibility, and thus edges that share the same set of giving or receiving nodes are no longer independent from each other. Second, the distribution of influence and susceptibility over network nodes is heterogeneous. It is also reported in (Aral and Walker, 2012) that influential individuals cluster in the network while susceptible individuals do not. Hence, a joint modeling of nodes’ influence and susceptibility is necessary. In a recent work, Aral and Dhillon (Aral and Dhillon, 2018) showed that failing to differentiate influence and susceptibility across network nodes caused an influence maximization algorithm to underestimate influence propagation by $21.7\%$ on average, for a fixed seed set size. To overcome the issue, some online influence maximization algorithms (Wen et al., 2017; Vaswani et al., 2017; Kalimeris et al., 2018) introduced edge-level features to help the modeling of influence probability. However, as revealed in (Aral and Walker, 2012), the factors affecting users’ influence and susceptibility may include age, gender, marital status and many other sensitive attributes, which can hardly be exhausted and are often prohibited under privacy constraints. Third, edge-level estimation of activation probability costs an algorithm both high computational complexity and sample complexity (Chen et al., 2016), given the number of edges in a network is usually significantly larger than the number of nodes. Utilizing the dependency in the influence probability across edges is expected to reduce the complexity.

In this paper, we propose to model the dependency of activation probabilities on the edges for online influence maximization. Specifically, we assume each network node’s influence and susceptibility are distinct and individually specified; and the activation probability on an edge is jointly determined by the giving node’s influence and receiving node’s susceptibility. This makes the activation probability matrix for the network low-rank. Then with such a low-rank structure, we propose a factorization based bandit solution to learn the latent influence factors and susceptibility factors on the nodes from the interactions with the environment. There are three important advantages of our factorization based bandit solution for online influence maximization. Firstly, it is able to capture the assortative mixing property of influence distribution in a network. Secondly, by directly learning the node-level parameters, the activation observation from one edge can be readily leveraged to other edges that share the same node. This reduces sample complexity for online learning. Third, comparing to existing online influence maximization solution with linear generation assumptions (Wen et al., 2017; Vaswani et al., 2017; Kalimeris et al., 2018), our solution does not depend on the availability of manually constructed edge- or node-level features; it learns the property of network nodes via factorization. A rigorous theoretical analysis on the upper regret bound of the proposed solution is provided, where we prove considerable regret reduction comparing to solutions that model the activation probability on edges independently. Extensive empirical evaluations on two large-scale networks confirmed the effectiveness of our proposed solution.

2. Related Work

The problem of influence maximization has been extensively studied in offline settings (Kempe et al., 2003; Chen et al., 2009, 2010), where the main focus is on computational efficiency in optimization. These solutions simply assume the influence model, i.e., the activation probabilities, can be specified by the network’s properties (e.g., node degree), or by a transmission parameter that is specified as constant, random or drawn from a uniform distribution, or by estimations from logged propagation data (Bourigault et al., 2016; Goyal et al., 2010; Netrapalli and Sanghavi, 2012; Saito et al., 2008). However, such an overly simplified influence model ignores many important properties of real network influence patterns, e.g., assortativity (Newman, 2002). To conquer these limitations, online influence maximization has been studied under different assumptions and settings. Chen et al. (Chen et al., 2013) and Wang and Chen (Wang and Chen, 2017) formulated it as a combinatorial bandit problem and proposed an upper confidence bound (auer2002using) based algorithm to estimate activation probabilities on a per-edge basis. Lei et al. (Lei et al., 2015) studied a different objective of maximizing the number of unique activated nodes across multiple rounds of activation. In (Vaswani et al., 2015), a bandit learning algorithm with both node-level (which nodes are eventually activated) and edge-level (who activated those nodes) feedback was developed. Methods similar to (Goyal et al., 2010; Netrapalli and Sanghavi, 2012; Saito et al., 2008) were used to map the node-level feedback back to edge-level activation probability estimation. Wen et al. (Wen et al., 2017) assumed the activation probability is a linear combination of edge-level features and an unknown global activation parameter, and proposed a linear contextual bandit solution. However, in practice it is very difficult to exhaustively specify the features for influence modeling on every edge, and many of those features are prohibited under privacy constraints, such as age, gender and martial status (Aral and Walker, 2012).

Our solution falls into another line of online influence maximization research that estimates influence parameters at the node level. Carpentier and Valko (Carpentier and Valko, 2016) proposed a minimax optimal algorithm under a local model of the influence spread, where a source node can only activate its neighbors without considering influence cascade. Vaswani et al. (Vaswani et al., 2017) proposed a diffusion model independent algorithm to estimate pairwise reachability between all pairs of nodes in a network. Olkhovskaya et al. (Olkhovskaya et al., 2018) developed an algorithm for node-level feedback with a theoretical guarantee. But their algorithm is only designed for graphs with specific structures. In general, previous works in this line of research do not explicitly separate influence and susceptibility of nodes, which causes seriously degraded estimation of influence propagation, as reported in (Aral and Dhillon, 2018). To the best of our knowledge, ours is the first to model the structural dependency in edge activation probabilities for online influence maximization. We assume the activation probability of an edge is jointly determined by the influence of its giving node and the susceptibility of its receiving node. We estimate the influence and susceptibility parameters of nodes via edge-level activation feedback, without the need of edge features. This directly leads to reduced computational complexity in model estimation and reduced regret in influence maximization.

3. Method

In this section, we discuss our developed factorization-based bandit solution for online influence maximization. We first illustrate the problem setup, then introduce the details of our solution, and conclude this section by comparing our solution with existing algorithms to demonstrate its unique advantages.

3.1. Problem Setup

3.1.1. Influence maximization

In an influence maximization problem, an input social network is modeled as a probabilistic directed graph $G=(\mathcal{V};\mathcal{E};\mathbf{P})$ with nodes $\mathcal{V}$ over a set of users, edges $\mathcal{E}$ for the set of directed connections between users, and activation probabilities $\mathbf{P}=(p_{e_{1}},p_{e_{2}},...,p_{e_{|\mathcal{E}|}})$ where $p_{e}\in[0,1]$ represents the probability that the receiving node of edge $e$ will be activated by the giving node on $e$. We describe how influence propagates from nodes to their network neighbors with a stochastic diffusion model $D$. The influence spread of the selected seed node set $\mathcal{S}$ with activation probability $\mathbf{P}$ is the expected number of nodes activated by $\mathcal{S}$ under the diffusion model $D$, denoted by $f_{D,\mathbf{P}}(\mathcal{S})$. Given the graph $G$, the IM problem aims to find a particular set $\mathcal{S}$ under a cardinality constraint $K$, which maximizes the influence spread $\mathcal{S}^{opt}=\operatorname*{arg\,max}_{|\mathcal{S}|\leq K}f_{D,\mathbf{P}}(\mathcal{S})$. This problem is known to be NP-hard (Nemhauser et al., 1978; Kempe et al., 2003), but it can be efficiently solved with approximations (Tang et al., 2014, 2015; Nguyen et al., 2016). In this paper, such influence maximization algorithms are treated as an oracle, which takes a given graph, seed set size and activation probability as input, and outputs an appropriate set of seeds. Define ${\mathcal{S}^{*}}=ORACLE(G,K,\mathbf{P})$ as the solution from the oracle. It serves as an $(\alpha,\gamma)$-approximation of $\mathcal{S}^{opt}$, where $f_{D,\mathbf{P}}(\mathcal{S}^{*})\geq\gamma f_{D,\mathbf{P}}(\mathcal{S}^{opt})$ with probability at least $\alpha$ (Chen et al., 2013). For example, we have $\gamma=1-\frac{1}{e}-\varepsilon$ and $\alpha=1-1/|\mathcal{V}|$ for most of influence maximization solutions (Kempe et al., 2003; Chen et al., 2009, 2010), where $e$ is the base of natural logarithm and $\varepsilon$ depends on the accuracy of their Monte-Carlo estimate.

3.1.2. Online influence maximization with bandit

Traditional works in influence maximization (Kempe et al., 2003; Chen et al., 2009, 2010) assume that in addition to the network structure, the algorithm either knows the per-edge activation probability or the probability can be specified from past propagation data (Goyal et al., 2010; Netrapalli and Sanghavi, 2012; Saito et al., 2008). However, as we discussed before this assumption imposes various limitations for the practical use of influence maximization solutions; and it is preferred for the learner to estimate the activation probabilities by directly interacting with the network, i.e., online influence maximization. In each round of online influence maximization, the learner needs to choose seeds to 1) maximize influence spread (i.e., exploitation), and 2) improve its knowledge of the activation probabilities via feedback (i.e., exploration). Multi-armed bandit framework is thus a natural choice to handle the exploration-exploitation trade-off for online influence maximization (Vaswani et al., 2015; Wen et al., 2017).

Formally, at each round $t$ during online influence maximization, the learner first chooses a seed node set ${\mathcal{S}}_{t}\in\mathcal{V}$ with cardinality $K$ by a predefined oracle based on its current knowledge from past observations: ${\mathcal{S}}_{t}=ORACLE(G,K,\hat{\mathbf{P}}_{t})$, where $\hat{\mathbf{P}}_{t}$ is the learner’s current estimate of activation probability $\mathbf{P}$. Influence then diffuses from the nodes in ${\mathcal{S}}_{t}$ according to the diffusion model $D$. For example, under the Independent Cascade (IC) diffusion model, this can be interpreted as the environment generates a binary response $y_{e,t}$ by independently sampling $y_{e,t}\sim\text{Bern}(p_{e})$ for each concerned edge $e$ in the resulting cascade. Note the environment is always using ground-truth activation probability $\mathbf{P}$. Then the learner receives the reward $f_{\text{IC},\mathbf{P}}({\mathcal{S}}_{t})$ which is the number of activated nodes by ${\mathcal{S}}_{t}$. We should emphasize that for any edge $e\in\mathcal{E}$, the learner observes the realization of $y_{e,t}$ if and only if its giving node is activated. To clarify, the active nodes deactivate at each time step, which makes it different from the setting in adaptive influence maximization (Golovin and Krause, 2011). With such edge-level feedback, the learner updates its estimate about the edge activation probability. The learner’s objective is to maximize the expected cumulative reward over a finite $T$ rounds of interactions.

3.2. A Factorization Bandit Solution

In most of existing bandit-based online influence maximization solutions, the learner estimates the activation probabilities on all edges independently (Chen et al., 2013, 2016), which cannot capture how the influence between nodes is formed and thus ignores the underlying structure behind the activation probabilities of edges. As suggested in a recent work (Aral and Dhillon, 2018), modeling nodes’ influence and susceptibility is vital for influence maximization. In this work, we explicitly model this influence structure by assuming activation probability $p_{e}\in[0,1]$ on edge $e$ can be decomposed into two $d$-dimensional latent factors on the giving node and receiving node, i.e.,

[TABLE]

where $g_{e}$ and $r_{e}$ denote the giving node and receiving node on edge $e$ respectively. $\mathrm{\bm{\theta}}_{v}\in\mathbb{R}^{d}$ is considered as the influence factor on node $v$ and $\bm{\beta}_{v}\in\mathbb{R}^{d}$ is the susceptibility factor on node $v$. This essentially imposes a low-rank structure assumption on the activation probability $\mathbf{P}$ and considers the activation probabilities as a reflection of influence and susceptibility properties of the connected nodes. It naturally captures the assortativity of the influence network: users who are more influential, i.e., the nodes that have larger values on $\mathrm{\bm{\theta}}_{v}$, trend to associate with larger activation probability on their neighboring node. The same for users who are more susceptible (e.g., measured by $\bm{\beta}_{v}$).

In this work, we assume the diffusion follows the independent cascade model (Kempe et al., 2003), and the edge-level feedback $y_{e,t}$ is observable to the learner. The estimation of activation probability $\mathbf{P}$ can thus be obtained by an online factorization of the sequential observations about edge activations in the network. As a result, instead of learning the edge activation probabilities directly, the complexity of which is $O(|\mathcal{E}|)$, our solution learns the underlying influence factor $\mathrm{\bm{\theta}}_{v}$ and susceptibility factor $\bm{\beta}_{v}$ for all the nodes. This reduces the model complexity to $O(d|\mathcal{V}|)$, which is considerably smaller than $O(|\mathcal{E}|)$, especially in a large and dense graph. To better illustrate our proposed learning solution, we introduce the following definitions about observed edges and observed nodes.

Definition 3.1 (Observed edge).

For any round $t$, a directed edge $e$ is considered as observed if and only if its start node is activated.

Note that an edge is observed does not necessarily mean the edge is active (i.e., not necessarily $y_{e,t}=1$). With the observed edges, a set of nodes can be classified as observed, which is defined as follows,

Definition 3.2 (Observed node).

A node $v$ is observed if and only if at least one of its giving-neighbor nodes is active.

Based on the above definition of observed edges and observed nodes, we can realize the relationship between observed edges and observed nodes: the giving and receiving nodes on an observed edges are both observed nodes. In this case, once an activation observation $y_{e,t}=1$ is obtained on edge $e$ at time $t$, statistics about the influence factor on its giving node $g_{e}$ and susceptibility factor on its receiving node $r_{e}$ can be updated. More importantly, all observed edges can be used for model update, which enables observation propagation to unobserved edges. We will discuss the benefit of this aspect with more details in Section 3.3.

Due to the coupling between $\mathrm{\bm{\theta}}$ and $\bm{\beta}$ imposed in the influence structure assumption in Eq (1), we appeal to a coordinate decent algorithm built on matrix factorization to estimate the latent influence factor $\mathrm{\bm{\theta}}_{v}$ and susceptibility factor $\bm{\beta}_{v}$ for each node $v\in\mathcal{V}$. Specifically, the objective function for our factor estimation can be written as follows,

[TABLE]

where $\tilde{\mathcal{E}}_{t}$ is the set of observed edges at time $t$, $\lambda_{1}$ and $\lambda_{2}$ are L2 regularization coefficients. The inclusion of L2 regularization term is critical to our solution in two folds. First, it makes the sub-problems in coordinate decent based optimization well-posed, so that we have closed form solutions for $\mathrm{\bm{\theta}}_{v}$ and $\bm{\beta}_{v}$ at each round. Second, it helps to remove the scaling indeterminacy between the estimates of $\mathrm{\bm{\theta}}_{v}$ and $\bm{\beta}_{v}$, and makes the $q$-linear convergence rate of parameter estimation achievable (Uschmajew, 2012; Wang et al., 2016, 2017). The closed-form estimation of $\mathrm{\bm{\theta}}_{v}$ and $\bm{\beta}_{v}$ with respect to Eq (2) at round $t+1$ can be obtained by $\hat{\mathrm{\bm{\theta}}}_{v,t+1}=\mathbf{A}_{v,t+1}^{-1}\mathbf{b}_{v,t+1}$ and $\hat{\bm{\beta}}_{v,t+1}=\mathbf{C}_{v,t+1}^{-1}\mathbf{d}_{v,t+1}$, in which,

[TABLE]

$\tilde{\mathcal{E}}_{i}^{v,\text{giving}}$ is the set of observed edges at round $i$ where node $v$ is the giving node, and accordingly $\tilde{\mathcal{E}}_{i}^{v,\text{receiving}}$ is the set of observed edges where node $v$ is the receiving node. $\mathbf{I}$ is a $d\times d$ identity matrix.

The estimated influence factors $\{\hat{\mathrm{\bm{\theta}}}_{v,t}\}_{v\in\mathcal{V}}$ and susceptibility factors $\{\hat{\bm{\beta}}_{v,t}\}_{v\in\mathcal{V}}$ give us an estimate of the edge activation probability by $\hat{p}_{e,t}=\hat{\mathrm{\bm{\theta}}}^{\mathsf{T}}_{g_{e},t}\hat{\bm{\beta}}_{r_{e},t}$ at round $t$. This represents the model’s current best knowledge about influence propagation in graph $G$, and therefore serves for the exploitation purpose. We also need a term to control exploration in online learning; and we appeal to the upper confidence bound (UCB) principle in this work (Auer, 2002), as the confidence interval of our factor estimation is readily available. In our solution, the uncertainty of activation probability estimation during online update comes from the uncertainty on both latent factors, i.e., $\lVert\hat{\mathrm{\bm{\theta}}}_{v,t}-\mathrm{\bm{\theta}}_{v}^{*}\rVert$ and $\lVert\hat{\bm{\beta}}_{v,t}-\bm{\beta}_{v}^{*}\rVert$, where $\mathrm{\bm{\theta}}_{v}^{*}$ and $\bm{\beta}_{v}^{*}$ are the ground-truth factors. Based on the closed form solution in our coordinate descend estimation, the estimation of activation probability $\hat{p}_{e,t}$ can be analytically bounded with a high probability as shown in the following lemma.

Lemma 3.3 (Confidence bound of influence probability estimation).

For any node $v$, and directed edge $e$, $\delta\in(0,1),q\in(0,1)$, with a probability at least $1-\delta$, we have,

[TABLE]

where $B^{\mathrm{\bm{\theta}}}_{v,t}=\alpha_{v}^{\mathrm{\bm{\theta}}}\lVert\hat{\mathrm{\bm{\theta}}}_{v,t}\rVert_{\mathcal{C}_{v,t}^{-1}}$, $B^{\bm{\beta}}_{v,t}=\alpha_{v}^{\bm{\beta}}\lVert\hat{\bm{\beta}}_{v,t}\rVert_{\mathbf{A}^{-1}_{v},t}$, $\alpha_{v}^{\mathrm{\bm{\theta}}}=\sqrt{\log\frac{\text{det}(\mathbf{C}_{v,T})}{\delta^{2}\text{det}(\lambda_{2}\mathbf{I})}}+\frac{\lambda_{2}(1-q)+2q}{\sqrt{\lambda_{2}}(1-q)}$, and $\alpha_{v}^{\bm{\beta}}=\sqrt{\log\frac{\text{det}(\mathbf{A}_{v,T})}{\delta^{2}\text{det}(\lambda_{1}\mathbf{I})}}+\frac{\lambda_{1}(1-q)+2q}{\sqrt{\lambda_{1}}(1-q)}$.

This lemma provides a tight high probability upper bound of the activation probability estimation on each edge, which enables us to perform efficient UCB-based exploration for model update. Denote the resulting upper confidence bound as $\bar{p}^{\prime}_{e,t}$, with $\bar{p}^{\prime}_{e,t}=\hat{\mathrm{\bm{\theta}}}_{g_{e},t}^{\mathsf{T}}\hat{\bm{\beta}}_{r_{e},t}+\text{CB}_{e,t}$ and $\text{CB}_{e,t}=B^{\mathrm{\bm{\theta}}}_{g_{e},t}+B^{\bm{\beta}}_{r_{e},t}+2q^{2t}$. As this upper bound might not be in a valid value as a probability, we use a projection operation to map it to the range of $[0,1]$ and get $\bar{p}_{e,t}=\text{Proj}_{[0,1]}(\bar{p}^{\prime}_{e,t})$. $\text{Proj}_{[0,1]}(x)$ is a function projecting its real value input $x$ to the range of $[0,1]$. More specifically, if the input $x$ is negative, the output is [math]; if $x$ is larger than $1$, the output is $1$; otherwise $x$ will be returned. Note that $\bar{p}^{\prime}_{e,t}$ is a high probability upper bound of $p_{e,t}$, i.e., $p_{e,t}\leq\bar{p}^{\prime}_{e,t}$. After this projection operation we can still guarantee that with a high probability $p_{e,t}\leq\bar{p}_{e,t}$. This is true because: 1). When $\bar{p}^{\prime}_{e,t}>1$, $\bar{p}_{e,t}=1$, which naturally guarantees $p_{e,t}\leq\bar{p}_{e,t}$; 2). When $\bar{p}^{\prime}_{e,t}<0$, $\bar{p}_{e,t}=0$, we have $p_{e,t}\leq\bar{p}^{\prime}_{e,t}<\bar{p}_{e,t}$. As an upper confidence bound of activation probability $\bar{p}_{e,t}$, it well balances the need of exploration and the need of exploitation (Auer, 2002). We thus directly feed $\bar{\mathbf{P}}_{t}=(\bar{p}_{e_{1},t},\bar{p}_{e_{2},t},....,\bar{p}_{e_{|\mathcal{E}|},t})$ at round $t$ into the ORACLE to solve one round of influence maximization, from which edge-level feedback on the observed edges will be obtained. Based on the observed edges, we get a set of observed nodes, and node-level influence factor $\mathrm{\bm{\theta}}_{v}$ and susceptibility factor $\bm{\beta}_{v}$ and their associated statistics can be updated based on their closed form solutions. We summarize the details of our factorization based online influence maximization algorithm in Algorithm 1, and name it as Influence Maximization with Factorization-Bandits, or IMFB in short.

3.3. Comparison with Existing Solutions

3.3.1. Node-level v.s., edge-level influence model estimation

In our proposed solution, we directly estimate the node-level influence and susceptibility factors by performing online factorization based on sequential edge-level observations. In this subsection, we illustrate the advantages of such a node-level parameter estimation by comparing with solutions that directly estimate edge-level parameters. One classic online influence maximization solution with edge-level parameter estimation is the CUCB algorithm (Wang and Chen, 2017). Comparing to CUCB, there are two important advantages in IMFB. First, reduced model complexity. In CUCB the edge activation probabilities are estimated independently across the edges, the complexity of which is $O(|\mathcal{E}|)$ (and it can become as large as $O(|\mathcal{V}|^{2})$ in the complete graph case). While the model complexity in IMFB is $O(d|\mathcal{V}|)$, which is considerably smaller, as the dimension of latent space should be much smaller than the number of nodes. Second, reduced sample complexity. Figure 1 shows an example of influence model update in CUCB and IMFB after one round of influence maximization. Suppose node $v_{0}$ is activated at this round, according to the definition of observed edges, $e_{v_{0},v_{1}}$ and $e_{v_{0},v_{2}}$ (edges with solid lines) are observed edges, and $e_{v_{3},v_{1}}$, $e_{v_{4},v_{1}}$, and $e_{v_{5},v_{1}}$ (edges shown with dash lines) are unobserved edges. In CUCB, shown in Figure 1 (a), its activation probability statistics will only be updated for those observed edges and no information is learned for those unobserved edges. In IMFB, by utilizing the fact that the activation on a particular edge is a reflection of the giving node’s influence and the receiving node’s susceptibility, observations from the observed edges can be propagated to the unobserved edges, as shown in Figure 1 (b). Specifically, because $\hat{\bm{\beta}}_{v_{1}}$ will be updated by this round’s observations, the estimation of $\hat{p}_{e_{v_{3},v_{1}}}$, $\hat{p}_{e_{v_{4},v_{1}}}$ and $\hat{p}_{e_{v_{5},v_{1}}}$ can all be improved for the next round of interaction. This directly accelerates the learning of influence structure. This advantage becomes more obvious and significant when the network is denser. Our theoretical analysis in the following section proves that when the network is fully connected, we can reduce the regret of IMFB comparing to CUCB by a factor of $\sqrt{|\mathcal{V}|}$. In addition, as we discussed earlier, independent edge-level estimations cannot guarantee to capture the important properties of influence propagation in real networks, e.g., assortativity. IMFB explicitly encode such structure into model learning, which can better realize the structural dependency in influence propagation.

3.3.2. Activation probability v.s., reachability

A recent work, DILinUCB (Vaswani et al., 2017), also obtains an $O(d|\mathcal{V}|)$ model complexity by performing node-level parameter estimation. But DILinUCB is conceptually very different from our IMFB. Instead of estimating the activation probability on edges from the interactions, DILinUCB directly estimates the reachability between all pairs of nodes (no matter if they are directly connected or not) from edge-level feedback via a linear contextual bandit model. There are two important limitations of it. Firstly, as mentioned in their paper, the proposed reachability based surrogate objective is only an heuristic approximation to the original online influence maximization problem. There is no theoretical guarantee about its approximation quality. Secondly, although its claimed diffusion model independent, their linear reachability assumption may violate the influence diffusing rules in some diffusion models, for example the independent cascading diffusion model.

4. Regret Analysis

Let $\mathcal{S}^{opt}$ be the optimal solution on an input graph, and $\mathcal{S}^{*}=\text{ORACLE}(G,K,\mathbf{P}^{*})$ be the solution from an oracle. Here we use $\mathbf{P}^{*}$ to denote the ground-truth activation probability in the network. For any $\alpha,\gamma\in[0,1]$, we say that ORACLE is an $(\alpha,\gamma)$-approximation oracle for a given $(G,K)$ if for any activation probability vector $\mathbf{P}$, $f_{\mathbf{P}}(\mathcal{S}^{*})\geq\gamma f_{\mathbf{P}}(\mathcal{S}^{opt})$ with probability at least $\alpha$. In this section, we simplify the notation of reward function $f_{D,\mathbf{P}}(\mathcal{S})$ to $f_{\mathbf{P}}(\mathcal{S})$, since we are consistently using independent cascade model.

To analyze the performance of online influence maximization algorithms with an approximation/randomized oracle, we look at an approximate performance metric – the scaled cumulative regret: $R^{\alpha\gamma}(T)=\sum_{i=1}^{T}\mathbb{E}[R_{i}^{\alpha\gamma}]$, in which $\alpha\gamma>0$ is the scale and $R_{t}^{\alpha\gamma}=f_{\mathbf{P}^{*}}(\mathcal{S}^{opt})-\frac{1}{\alpha\gamma}f_{\mathbf{P}^{*}}(\mathcal{S}_{t})$. Based on this regret definition, we have Theorem 4.1 specifying the regret upper bound of IMFB, which utilized two important proprieties of the reward function $f_{\mathbf{P}}(\mathcal{S})$ in influence maximization problem: monotonicity and 1-Norm bound smoothness (Wang and Chen, 2017).

Theorem 4.1.

Assuming that the ORACLE is an $(\alpha,\gamma)$-approximation algorithm, we have the following upper regret bound in IMFB,

[TABLE]

in which $B$ is the bounded smoothness constant and it can be proved that $B\leq|\mathcal{V}|$ in influence maximization problem (Wang and Chen, 2017). $D^{\text{out}}$ and $D^{\text{in}}$ are the maximum out-degree and in-degree of graph $G$. $|\mathcal{V}^{\text{giving}}|$ and $|\mathcal{V}^{\text{receiving}}|$ are the maximum number of observed giving nodes and receiving nodes at any time point during the interactions.

Proof sketch of Theorem 4.1: According to the regret definition, the expected regret at time $t$ can be bounded by,

[TABLE]

in which the second inequality is based on the monotonicity of the reward function and the ORACLE’s seed node selection. The third inequality is based on the 1-Norm bounded smoothness of the reward function and $B$ is the bounded smoothness constant (Wang and Chen, 2017). Then combining the activation probability upper bound provided in Lemma 3.3, the conclusion in Theorem 4.1 can be obtained. Due to space limit, detailed proof of this theorem and related technical lemmas will be provided in the supplement of this paper.

Analysis of Theorem 4.1: Since $|\mathcal{V}^{\text{receiving}}|$, $|\mathcal{V}^{\text{giving}}|$, $D^{\text{out}}$, and $D^{\text{in}}$ can all be simply upper bounded by $|\mathcal{V}|$, the worst case upper regret bound of IMFB is $O(d|\mathcal{V}|^{\frac{5}{2}}\sqrt{T})$, which is still better than the upper regret bound most online influence maximization algorithms: in a complete graph, their upper regret bound for IMLinUCB (Wen et al., 2017) and CUCB (Wang and Chen, 2017) are $O(d|\mathcal{V}|^{3}\sqrt{T})$ and $O(|\mathcal{V}|^{3}\sqrt{T})$ respectively.

5. Experiments

In this section, we performed extensive empirical evaluations on two real-world networks and compare the proposed solution with a set of state-of-the-art online influence maximization baselines.

5.1. Datasets

We evaluate our proposed IMFB algorithm on two real-world datasets: NetHEPT111https://snap.stanford.edu/data/cit-HepTh.html (with 27,770 nodes, and 352,807 ) and Flickr222https://snap.stanford.edu/data/web-flickr.html (with 12,812 nodes and 137,986 edges). On the NetHEPT dataset, the ground-truth influence factors and susceptibility factors on the nodes are sampled from a uniform distribution $U(0,0.1)$ and then normalized by their L2 norms. The activation probabilities on the edges are generated according to Eq (1). The average edge activation probability on this resulting dataset is 0.053. The Flickr dataset has a very skewed degree distribution, which makes it a very easy task for all the online influence maximization algorithms if the influence factors and susceptibility factors of all the nodes are sampled from the same distribution. In order to make the learning task on this dataset more challenging, we need to make the ‘soft degree’ distribution on this network more balanced. Here we define a node’s ‘soft degree’ as the summation of its activation probability on all its connected out-going edges. In order to achieve this, we sample influence factors and susceptibility factors from 10 uniform distributions with different expectations, including $U(0,0.1),U(0.1,0.2),...,U(0.9,1.0)$, and then normalize them using their L2 norms respectively. We group the nodes into 10 groups according to their ‘hard degree’ (here a node’s original degree is referred as ‘hard degree’). We assign the groups of nodes with higher hard degree to the distribution that has a lower expectation, such that we have a much more balanced soft degree distribution on this network. This makes the learning of activation probabilities more challenging. The average edge activation probability on this resulting dataset is 0.065.

In all following experiments, the dimensionality for the latent influence and susceptibility factor is set to 20, and seed node size $K$ is set to 300 by default.

5.2. Baseline Algorithms

We empirically evaluate our proposed IMFB algorithm against the following state-of-the-art online IM algorithms.

$\bullet$ CUCB (Chen et al., 2013; Wang and Chen, 2017): it estimates the activation probabilities on the edges independently and uses upper confidence bound of the estimation to balance exploitation and exploration.

$\bullet$ $\epsilon$-greedy: it estimates the activation probabilities on the edges independently, and uses $\epsilon$-greedy (Sutton et al., 1998) to balance exploitation and exploration.

$\bullet$ IMLinUCB (Wen et al., 2017): it learns an edge-level bandit model and estimates the activation probabilities with given edge-level features. In our experiment, we test two variants of IMLinUCB depending on how the edge-level features are constructed: (1). We directly use the ground-truth influence and susceptibility factors $\mathrm{\bm{\theta}}^{*}$ and $\bm{\beta}^{*}$ to generate a set of high-quality features for it. For each edge $e$, we take outer product on $\mathrm{\bm{\theta}}_{g_{e}}^{*}$ and $\bm{\beta}_{r_{e}}^{*}$ and reshape it to a $d^{2}$-dimensional vector. We need to emphasize that since $\mathrm{\bm{\theta}}_{g_{e}}^{*}$ and $\bm{\beta}_{r_{e}}^{*}$ are directly used to generate the ground-truth activation probability, this feature setting can lead to the best possible version of IMLinUCB, i.e., it only needs to recognize the diagonal terms in the outer product. (2). We follow the setting proposed in (Wen et al., 2017), where we first use node2vec (Grover and Leskovec, 2016) to create node embedding vectors and use an element-wise product between two nodes of an edge to get the edge features. This variant of IMLinUCB is referred to as IMLinUCB-Node2Vec in the following discussion.

$\bullet$ DILinUCB (Vaswani et al., 2017): is a model-independent contextual bandit algorithm for online influence maximization. DILinUCB requires features on the receiving nodes as input to estimate the reachibility between node pairs. We provide ground-truth susceptibility factor $\bm{\beta}_{v}^{*}$ for $v\in\mathcal{V}$ as input of node features to DILinUCB.

For IMFB and all baselines except DILinUCB, the DegreeDiscountIC algorithm proposed in  (Chen et al., 2009) is used as the influence maximization oracle. For DILinUCB, since the original influence maximization objective function is replaced with a heuristic surrogate function, the greedy oracle suggested in their solution (Vaswani et al., 2017) is used here. In all our experiments, we run all algorithms over 200 rounds and report the averaged results from 5 independent runs.

5.3. Comparison on Influence Maximization

We first compare different algorithms’ performance in maximizing influence during online learning. For better visibility, all subfigures in Figure 2 share the same set of legend.

In Figure 2 (a) & (b), we report the real-time average reward and variance (showed by the shaded area) of the collected reward from different algorithms on NetHEPT and Flickr dataset respectively. From this result, we first observe that IMFB shows significant improvement over all the baselines on both datasets. We also observe that on both datasets DILinUCB performs consistently the worst, which is expected because of the heuristic reachability assumption.

On the NetHEPT dataset shown in Figure 2 (a), we can observe that the learning process of CUCB and $\epsilon$-greedy are much slower than the other solutions. This observation verified our claim about the necessity of explicitly modeling the influence structure: when the edge activation probabilities are independently modeled (e.g., in CUCB and $\epsilon$-greedy), the learner can only update and improve its estimation quality on the observed edges. This makes the learning very slow, especially on this relative denser dataset. Although uncertainty of the estimation can help avoid local optimal, it may also take time to recognize the globally optimal seed nodes, especially when a lot of edges need exploration. While for IMFB and both variants of IMLinUCB, since they can propagate observations through node-level parameter estimation (in IMFB) or shared edge parameter estimation, the learning process is much faster.

On the Flickr dataset shown in Figure 2 (b), IMLinUCB performs even worse than CUCB. The potential reason for this bad performance is that IMLinUCB is over-parameterized, which slows down its learning convergence. For example, among all provided $d^{2}$ features, only $d$ of them are effective. The algorithm needs a lot more observations to recognize the relevance of those features. While at the same time, since the dataset is relatively sparser, i.e., the number of observed edges are relatively smaller in each round of interaction, which makes IMLinUCB’s learning process even slower. In addition, we can also notice that IMLinUCB-Node2Vec is one of the worst performing algorithms. This indicates IMLinUCB is very sensitive to the quality of manually constructed edge-features.

Based on our observations in Figure 2 (a) & (b), we designed a new setting on the Flickr dataset to better illustrate the importance and advantages of node-level parameter estimation in our proposed IMFB. On the Flickr dataset, we first categorize the nodes into two types: Type 1 includes nodes that have very high hard degrees; and Type 2 includes nodes that have relatively low hard degrees. Then we controlled the generation of influence and susceptibility factors to make the out-going edges from Type 1 nodes associated with extremely low activation probabilities. The purpose of this design is to make the average soft degree in Type 1 nodes small. And for Type 2 nodes, we control the activation probability generation to make their associated edges’ activation probability extremely large. And we remove some of the edges (but not all of them) connecting these two types of nodes to further control the degree distribution.

At the beginning of online influence maximization, when a learner’s estimation about activation probabilities is not accurate, hard degree plays a dominating role in the ORACLE’s decision. Hence, on this manipulated Flickr network, the ORACLE trends to start with Type 1 nodes (because their hard degree is very high) but the optimal seed nodes are more likely to concentrate on Type 2 nodes, where the average activation probability is higher. In this case, if there is no observation prorogation among edges, it would take more rounds of interactions for a learner to realize that Type 2 nodes are actually better. In IMFB, even though at the beginning stage it would also start from Type 1 nodes, because of its node-level parameter estimation, observations from observed edges can be effectively propagated to those unobserved edges. If those unobserved edges are connected to Type 2 nodes (because we did not remove all the edges connecting Type 1 and Type 2 nodes), estimation about the Type 2 nodes and their connected edges can thus be improved. This observation propagation to unobserved edges helps IMFB learn much faster than the other baselines. This benefit is also illustrated in Figure 1 of Section 3.3. Empirical evaluation of average reward under this setting can be found in Figure 2 (c), which verified our claimed benefit on IMFB: On this dataset, IMFB obtained much more significant improvement: it obtained $230\%$ improvement over CUCB and $\epsilon$-greedy.

5.4. Influence Model Estimation Quality

In order to better understand the algorithms’ effectiveness in activation probability learning, we compared the estimation quality of IMFB with related baselines on both datasets in Figure 3 with solid lines (using the right-side of y-axis). Specifically, we calculated the absolute difference between the estimated activation probability $\hat{p}_{e,t}$ and ground-truth probability $p_{e}^{*}$ over observed edges. We omit $\epsilon-$greedy and DILinUCB here since $\epsilon-$greedy’s performance is very similar to CUCB, and DILinUCB does not directly estimate the activation probabilities.

From the results on NetHEPT in Figure 3 (a), we can observe that, in addition to IMFB, the estimation error of activation probabilities in IMLinUCB also converge very fast but still worse than IMFB. IMLinUCB-Node2Vec has a much worse activation probability estimation than all the other baselines. The unsatisfactory activation probability estimation in CUCB again verified our motivation of joint edge level parameter estimation. We can see that at around round 85, there is a sudden jump of its activation probability estimation quality. This is because at that round CUCB starts to explore edges that were rarely observed before. For those edges, CUCB has to learn the activation probability from scratch, which leads to a sudden drop and a large variance in its estimation. This observation directly verify phenomenon shown in the illustration example in Figure 1, where we showed CUCB can only improve the estimation of activation probabilities on edges that are directly observed.

On the Flickr dataset in Figure 3 (b), we observe similar estimation error pattern in CUCB at round interaction 15. Since this happens much earlier than that on the NetHEPT dataset, the performance of CUCB is not greatly affected after all. And on this dataset, IMFB converges the fastest and converges to an error that is much smaller than any other baselines’. Results on both these two dataset empirically confirmed the reduced sample complexity of IMFB.

In addition, we also visualize the average L2 difference between the estimated factors in IMFB and the ground-truth factors in the environment, i.e., $\lVert\hat{\mathrm{\bm{\theta}}}_{v,t}-\mathrm{\bm{\theta}}^{*}_{v}\rVert_{2}$ and $\lVert\hat{\bm{\beta}}{v,t}-\bm{\beta}^{*}_{v}\rVert_{2}$, on observed nodes $v$. The results on both datasets are shown in Figure 3 (a) & (b) with dotted lines using the left side of the y-axis. We can find that within 25 iterations, IMFB’s estimation error in both $\mathrm{\bm{\theta}}$ and $\bm{\beta}$ reduces. As a result, its estimated activation probability $\hat{p}_{e,t}$ quickly converges to ground-truth probability, which corresponds to the convergence of the real time reward. The good estimation quality of activation probability in IMFB coincides our analysis in Lemma 3.3 regarding the confidence bound of activation probability estimation.

5.5. Robustness in Influence Model Learning

5.5.1. Dimensionality Misspecification

In our proposed solution, dimensionality of the latent influence factors and susceptibility factors need to be specified as the input. On the tested datasets, the dimensionality is set to 20 when generating the ground-truth activation probabilities. However, such information would be unknown to a learner in practice. Thus it is important to study the robustness of our proposed IMFB algorithm when dimensionality of latent factor is misspecified. Here we use IMFB-$d^{\prime}$ to denote IMFB with latent dimension size $d^{\prime}$. We vary $d^{\prime}$ to study IMFB’s robustness to dimensionality misspecification. Average reward on NetHEPT and Flickr dataset are reported in Table 1. We also include results from all the other baselines for comparison. From this result we can clearly find that: firstly IMFB is very robust to dimensionality misspecification, specifically when $d^{\prime}$ is set within a reasonable range. For example when $d^{\prime}$ is in the range of 5 to 60, given the ground-truth dimension size is 20, the variance in the performance of IMFB is very small. Even when $d^{\prime}$ is set to 1, IMFB can still maintain competitive performance. Secondly, by comparing IMFB with all other baselines, we can see that even when the dimensionality is misspecified, IMFB can still significantly outperform all the baselines. This suggests specifying the correct influence structure is more important for online influence maximization.

5.5.2. Different Environment Settings

To examine our algorithm’s robustness to different environment setups, we performed evaluation on the Flickr dataset by varying several important parameters of the environment, and report the results in Table  2.

$\bullet$** Number of seed nodes $K$.** We vary the number of seed nodes $K$ from 50 to 1,000. We observe that for different settings of $K$, IMFB consistently performs better than other baselines.

$\bullet$** Noise level $\eta$.** Previous works in online influence maximization assumes the activation of edge $e$ is controlled by a fixed ground-truth activation probability $p_{e}$. In order to get a sense of our algorithm’s robustness to additional noise on the edge activation, at each round of interaction we add an additional noise term $\eta$ on the ground-truth activation probabilities and then project the resulting noisy activation probabilities to $[0,1]$. From Table  2 we observe that IMFB is robust to different noise levels and outperforms other baselines even when the noise scale is very large.

$\bullet$** Activation probability scale coefficient $c$.** To simulate some practical scenarios where the graph is generally more active or inactive, we re-scale the activation probability $p_{e}$ in our default setting by a constant coefficient $c$, i.e., activate edge $e$ with probability $c\cdot p_{e}$ , where $c$ is varied from $0.5$ to $2.0$. According to the results in Table  2, our proposed IMFB is robust to the choice of $c$ and generally improves over other baselines. We also observe that when $c=2.0$ the difference between all algorithms becomes much smaller. This is expected: when $c$ is large, the network is very active in every node, which makes the number of observations much larger which reduces necessity of observation propagation.

6. Conclusions & Future Work

Motivated by the network assortative mixing property, we decompose the edge activation probability into giving node’s influence and receiving node’s susceptibility, and therefore perform node-level influence model estimation for better model complexity and generalization in online influence maximization. We provide rigorous theoretical analysis of IMFB, and show considerable regret reduction comparing with several state-of-the-art online maximization solutions. We also empirically evaluate on two real-world network datasets and confirm the effectiveness and robustness of IMFB.

Our factorization based solution provides a flexible framework to introduce more desired properties, e.g., homophily and simultaneity, to influence modeling. Currently, we have assumed the availability of edge-level feedback from influence maximization. But in practice node-level feedback is more prevalent, we can incorporate more sophisticated inference methods based on our factor models to learn from such feedback.

7. Acknowledgements

We thank the anonymous reviewers for their insightful comments. This work was supported in part by National Science Foundation Grant IIS-1553568 and IIS-1618948 and Bloomberg Data Science PhD Fellowship. Dr. Wei Chen is partially supported by the National Natural Science Foundation of China (Grant NO.61433014).

8. Supplement

8.1. Implementation of IMFB

The implementation of our proposed solution IMFB is publicly available at https://github.com/Matrix-Factorization-Bandit/IMFB-KDD2019.

8.2. Proof of Theorem 4.1 and related lemmas

To carry out our analysis, we make the following mild conditions on the expected reward $f_{\mathbf{P}}(\mathcal{S})$. Note that these two conditions are the same as stated in (Wang and Chen, 2017).

• •

Monotonicity. The expected reward of playing any super arm $S\in\mathbf{S}$ is monotonically non-decreasing with respect to the expectation vector, i.e., if for all $i\in[m]$ $p_{i}\leq p_{i}^{\prime}$ , we have $f_{\mathbf{P}}(S)\leq f_{\mathbf{P}^{\prime}}(S)$ for all $S\in\mathbf{S}$, in which $\mathcal{S}$ is the set of all candidate super arms.

• •

1-Norm Bounded Smoothness. We say that a CMAB-T (Combinatorial Multi-Armed Bandit with probabilistically Triggered arms) satisfy 1-norm bounded smoothness, if there exists a bounded smoothness constant $B\in\mathbb{R}^{+}$ such that for any two distributions with expectation vectors $\mathbf{P}$ and $\mathbf{P}^{\prime}$, and any action $\mathcal{S}$, we have $|f_{\mathbf{P}}(\mathcal{S})-f_{\mathbf{P}^{\prime}}(\mathcal{S})|\leq B\sum_{i\in\tilde{\mathbf{S}}}|p_{i}-p_{i}^{\prime}|$, where $\tilde{\mathcal{S}}$ is the set of arms that are triggered by $\mathcal{S}$.

Proof of Theorem 4.1.

According to the definition of scaled regret, we have

[TABLE]

where the expectation is over the possible randomness of $\mathcal{S}_{t}$ since the oracle might be a randomized algorithm. Notice that the randomness coming from the edge activation is already taken care of in the definition of $r$ function. For any $t\leq T$, we define event $\xi_{t}$ as,

[TABLE]

and $\bar{\xi}_{t-1}$ as the complement event of $\xi_{t-1}$. Hence we have,

[TABLE]

in which the second inequality is based on the monotonicity of the expected reward function (since we proved that with high probability each element in $\bar{\mathbf{P}}_{t}$ is an upper bound of the coressponding element in $\mathbf{P}^{*}$. ), and the third inequality is based on the oracle’s seed node selection strategy.

By the 1-Norm bounded smootheness of the reward function as stated in Condition 2, we have,

[TABLE]

in which $\tilde{\mathcal{E}}_{t}$ is the set of observed edges at interaction $t$ (i.e. the set of triggered edges by $\mathcal{S}_{t}$ at time $t$ if map it to the 1-Norm Bounded Smoothness condition).

Substituting Eq (11) to Eq (10) and combining the definition of cumulateive regret we have,

[TABLE]

According to a proof similarly as in Lemma 2 of (Wen et al., 2017), the term $\mathbb{P}(\bar{\xi}_{t-1})(L-K)$ can be bounded by $1$. Then combining the conclusion in Lemma 8.1, we have,

[TABLE]

∎

Lemma 8.1.

Denote $\mathcal{V}_{S_{t}}^{\text{receiving}}$ as the set of observed giving nodes at time $t$ under seed node set $\mathcal{S}_{t}$, $\mathcal{V}_{S_{t}}^{\text{receiving}}$ as the set of receiving nodes at time $t$ under seed node set $\mathcal{S}_{t}$, we have with high probability,

[TABLE]

Proof of Lemma 8.1.

[TABLE]

in which $\alpha^{\bm{\beta}}=\max_{v\in\mathcal{V}}\alpha_{v}^{\bm{\beta}}$ and $\alpha^{\mathrm{\bm{\theta}}}=\max_{v\in\mathcal{V}}\alpha_{v}^{\mathrm{\bm{\theta}}}$.

[TABLE]

in which $|\mathcal{V}^{\text{giving}}|$ is the maximum number of observed giving nodes at any time point, and the last inequality is based on Lemma 8.2. and similarly we have,

[TABLE]

in which $|\mathcal{V}^{\text{receiving}}|$ is the maximum number of observed receiving nodes at any time point, and the last inequality is based on Lemma 8.2.

Then we obtain that,

[TABLE]

in which $\alpha^{\bm{\beta}}=\sqrt{d\log(\frac{d+TD^{\text{out}}}{d\delta^{2}\lambda_{1}})}+\frac{\lambda_{1}(1-q)+2q}{\sqrt{\lambda_{1}}(1-q)}$, and $\alpha^{\mathrm{\bm{\theta}}}=\sqrt{d\log(\frac{d+TD^{\text{in}}}{d\delta^{2}\lambda_{2}})}+\frac{\lambda_{2}(1-q)+2q}{\sqrt{\lambda_{2}}(1-q)}$ and $|\mathcal{E}_{O}|$ is the maximum number of observed edges at any interaction up to time $T$.

∎

Lemma 8.2.

Denote $\mathcal{V}_{S_{t}}^{\text{giving}}$ as the set of observed giving nodes at time $t$ under seed node set $\mathcal{S}_{t}$, $\mathcal{V}_{S_{t}}^{\text{receiving}}$ as the set of receiving nodes at time $t$ under seed node set $\mathcal{S}_{t}$, we have

[TABLE]

and

[TABLE]

Proof of Lemma 8.2.

The proof of this lemma is similar as Lemma 1 in (Wen et al., 2017). ∎

Proof of Lemma 3.3.

The proof of inequalities $\lVert\hat{\mathrm{\bm{\theta}}}_{v,t}-\mathrm{\bm{\theta}}_{v}^{*}\rVert_{\mathbf{A}_{v,t}}\leq\alpha_{v}^{\bm{\beta}}$ and $\lVert\hat{\bm{\beta}}_{v,t}-\bm{\beta}_{v}^{*}\rVert_{\mathbf{C}_{v,t}}\leq\alpha_{v}^{\mathrm{\bm{\theta}}}$ are technically similar as the proof in Lemma 1 in (Wang et al., 2016), and thus omitted here. For the simplicity of notations, we denote the head node and tail node of edge $e$ as $u$ and $v$ respectively.

[TABLE]

in which $q$ is a constant in the range of $(0,1)$, and the second inequality is based on the q-linear convergence rate of parameter estimation (Uschmajew, 2012; Wang et al., 2016). ∎