Quantized Distributed Online Projection-free Convex Optimization
Wentao Zhang, Yang Shi, Baoyong Zhang, Kaihong Lu, Deming Yuan

TL;DR
This paper introduces a quantized distributed online optimization algorithm for convex problems over multi-agent networks, balancing communication efficiency and convergence performance through a novel quantization approach.
Contribution
It develops a new quantized algorithm for distributed online convex optimization that reduces communication costs while maintaining theoretical convergence guarantees.
Findings
Achieves dynamic regret bounds under different quantizer settings.
Demonstrates trade-off between quantization level and convergence performance.
Validates results with simulations on distributed linear regression.
Abstract
This paper considers online distributed convex constrained optimization over a time-varying multi-agent network. Agents in this network cooperate to minimize the global objective function through information exchange with their neighbors and local computation. Since the capacity or bandwidth of communication channels often is limited, a random quantizer is introduced to reduce the transmission bits. Through incorporating this quantizer, we develop a quantized distributed online projection-free optimization algorithm, which can achieve the saving of communication resources and computational costs. For different parameter settings of the quantizer, we establish the corresponding dynamic regret upper bounds of the proposed algorithm and reveal the trade-off between the convergence performance and the quantization effect. Finally, the theoretical results are illustrated by the simulation of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDistributed Control Multi-Agent Systems · Energy Efficient Wireless Sensor Networks · Advanced Optimization Algorithms Research
Quantized Distributed Online Projection-free Convex Optimization
Wentao Zhang
Yang Shi
Baoyong Zhang
Kaihong Lu
Deming Yuan *Corresponding author: Deming Yuan.*W. Zhang, B. Zhang, D. Yuan are with School of Automation, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu, P. R. China (e-mail: [email protected], [email protected], [email protected]).Y. Shi is with the Department of Mechanical Engineering, University of Victoria, Victoria, BC V8W 2Y2, Canada (e-mail: [email protected]).K. Lu is with the College of Electrical Engineering and Automation, Shandong University of Science and Technology, Qingdao 266590, China (e-mail: [email protected])
Abstract
This paper considers online distributed convex constrained optimization over a time-varying multi-agent network. Agents in this network cooperate to minimize the global objective function through information exchange with their neighbors and local computation. Since the capacity or bandwidth of communication channels often is limited, a random quantizer is introduced to reduce the transmission bits. Through incorporating this quantizer, we develop a quantized distributed online projection-free optimization algorithm, which can achieve the saving of communication resources and computational costs. For different parameter settings of the quantizer, we establish the corresponding dynamic regret upper bounds of the proposed algorithm and reveal the trade-off between the convergence performance and the quantization effect. Finally, the theoretical results are illustrated by the simulation of distributed online linear regression problem.
I INTRODUCTION
In recent years, online distributed convex optimization has received ever-increasing attention from researchers because of its wide applications in many areas, such as machine learning, sensor networks, smart grids, etc.; see, e.g., [1, 2, 3, 4, 5, 6]. In such an online optimization problem with constraint sets, various algorithms with projection operations have been developed, such as distributed online gradient descent [7, 8]. However, for some high-dimensional and complex constrained optimization scenarios including multiclass classification [9] and matrix completion [10, 11], projection operations incur a heavy computational burden. On the contrary, projection-free algorithms have impressive advantages essentially due to the use of a linear oracle.
In [9], Zhang et al. earlier proposed an online distributed projection-free algorithm and established the static regret upper bound as . The works [12, 13, 14] further analyzed the static regret of some variants based on projection-free methods. In [15] and [16], the dynamic regret bounds were studied in distributed online projection-free algorithms under convex and nonconvex conditions, respectively. Dynamic regret is a more stringent metric than static regret due to its dynamic reference sequence. However, the communication channels between agents in [15], [16] are assumed to be perfect. In most applications, the communication channels often have limited bandwidth or capacity, especially for the cases with scarce communication bandwidth or capacity [17].
It is worth mentioning that quantized communication as a communication pattern can effectively reduce the number of communicated bits to achieve the saving of communication resources [17] through transmitting the information quantized by a dedicated quantizer. Currently, some distributed optimization algorithms with quantized communication have been developed [18, 19, 20, 21, 3, 22, 23, 24, 25], etc. In [18], Xiong et al. investigated the quantization effects on the convergence performance of the distributed quantized mirror descent algorithm. The works [19, 20, 21] analyzed the quantized distributed off-line optimization algorithms based on subgradient and inexact proximal-gradient methods, respectively. Doan et al. [22] considered a distributed off-line two-time-scale stochastic approximation algorithm under random quantization and established the almost sure convergence to the optimal solution for both convex and strongly convex loss function. In [23], Li et al. investigated the quantized distributed subgradient optimization algorithm with the dynamic encoding and decoding frameworks and proved that consensus optimization could be achieved under some mild conditions. Further, for the online distributed optimization problem, Yuan et al. [3] proposed a distributed online bandit algorithm under quantized communication and established the static regret. Up to now, there are few research results considering distributed online optimization scenarios under quantized communication. The above analysis of the related literature and the state of the art motivates us to investigate the dynamic regret of distributed online projection-free algorithm under quantized communication, and the effect of quantizer parameters on regret bounds.
The main contributions of this work are two-fold. Firstly, motivated by [15] and [3], we develop a quantized distributed online projection-free optimization (Q-DOPFO) algorithm for solving the distributed online constrained optimization problem over a multi-agent network. Meanwhile, the proposed algorithm saves the communication resources and computational costs as compared to the algorithms with real-valued data and projection operations, respectively. Secondly, for different parameter settings of the quantizer, we establish the corresponding dynamic regret upper bounds of the proposed algorithm and reveal the trade-off between the convergence performance and the quantization effect. In particular, when the knowledge of is known, the optimal bound can be achieved under proper parameter settings, where and represent the total time, function variation and gradient variation, respectively.
The remainder of the paper is organized as follows. The problem statement, quantizer description, and necessary assumptions are presented in Section II. Section III shows the algorithm design and convergence results. Sections IV and V give simulation examples and conclusion, respectively.
Notation: represents the Euclidean space with dimensions. denotes . denotes the Euclidean norm of a vector . represents the round up function. The boundary of a set is denoted as . The element in the -th row and -th column of matrix is denoted as . denotes the -th element of vector . is the closed Euclidean ball with a center point of origin and a radius of .
II Problem Formulation
II-A The Optimization Problem
Consider a directed time-varying network that consists of agents, where , denotes the edge set. In the network, agent can receive the information from the agents in its neighbor sets . denotes the weighted matrix and satisfies double stochasticity, i.e., , , where . There exists a constant such that holds when , and otherwise. The distributed online optimization problem is described as follows:
[TABLE]
where , the function is convex over the convex and compact set . Agents in the network cooperate to search for the global optima of Problem (1) through local computation and information exchange with neighbor agents. Generally, the metric defined in (2) is used to measure the algorithm performance, which represents the difference between the cumulative cost of the agent over time and the cumulative cost at benchmark sequence .
[TABLE]
where . Due to this varying benchmark , dynamic regret is more stringent than static regret and has wider application scenarios, such as target tracking. It is well known that the upper bound of (2) generally depends on the regularity of the optimization problem. Considering this fact, we define the following function variation and gradient variation .
[TABLE]
where , .
Our objective is to design a distributed online algorithm with quantized communication for Problem (1) that achieves sublinear dynamic regret of every agent .
II-B Random Quantizer
In this section, the following random quantizer is introduced to ensure that each agent in the network uses its quantized information to communicate with its neighbors.
Definition 1** ([3])**
* is the time-varying random quantizer of a vector if it satisfies that*
[TABLE]
where denotes a quantization resolution that is dependent on the qunantization levels and the dimension .
Remark 1
Several common quantizers are naturally special cases of this random quantizer, such as randomized gossip [26], rescaled unbiased estimators [26], stochastic -level quantization [3], probabilistic quantizer [18]. We show the probabilistic quantizer in [18] as an example. Denote , where . Then, for , we have
[TABLE]
where and are the round up and down to the nearest integer multiple of , respectively. It is not hard to note that the probabilistic quantizer satisfies Definition 1 with .
Remark 2
According to Definition 1, has a wide range of values and when its value is smaller, the quantized data is closer to the real-value data. Note that large values of are allowed at the early stages of the running algorithm, which means that the quantized data at this stage is coarser and less precise than the real-value data. In order to achieve the sublinear dynamic regret, a sublinearly convergent sequence over time is desired and necessary, which can be verified in the following sections.
II-C Some Assumptions
Some necessary assumptions are needed to facilitate the following algorithm development.
Assumption 1
The union is strongly connected for some positive integer and every integer .
Assumption 2
The constraint set is convex and compact and satisfies that .
Assumption 3
The function is -Lipschitz, i.e., , , where is a known positive constant.
Assumption 4
The gradient is -Lipschitz, i.e., , which is equivalent to .
Remark 3
Assumptions 1-3 are common in the literature (see [27, 1], [11, 28], etc.) on centralized and distributed optimization. The purpose of assuming is to ensure that the variance of the random quantizer is bounded, i.e., , which is a necessary precondition. It is worth noting that Assumption 3 implies according to Lemma 2.6 in [29].
III Algorithm Design and Convergence Analysis
III-A Algorithm Q-DOPFO
In this section, we develop Algorithm Q-DOPFO, which is illustrated in Algorithm 1. The key ingredients of the proposed algorithm include: 1) the quantized data and , instead of real-valued data, are utilized to perform consensus steps; 2) gradient tracking technique is introduced to correct the gradient change of loss function by using the global gradient estimation instead of individual agent gradients; 3) the decision variable is updated through a linear step. It is worth noting that the use of the random quantizer and projection-free oracle in the proposed algorithm can effectively save communication and computing resources of multi-agent systems.
In some extreme situations, such as at time , may occasionally violate the constraint set due to the quantizer. However, because of the variability of over the time and the randomness of the quantizer, usually does not always violate set . To ensure that the updated decipsion is always feasible, we require that the quantized state is in set for all , i.e. the step 4 of Algorithm 1.
III-B Main Convergence Results
In this section, some lemmas and the bound of dynamic regret defined in (2) for Algorithm 1 are established. To facilitate the analysis, we define as follows the transition matrix , for all , the running average vectors , , the quantization errors and the difference of quantized gradient .
[TABLE]
Lemma 1
Let the decision sequence be generated by Algorithm 1. Then, under Assumptions 1 and 2, we have for that
[TABLE]
where .
Lemma 2
Let the sequence be generated by Algorithm 1. Then, under Assumptions 1 and 4, we have for any that
[TABLE]
where .
Theorem 1
Let the decision sequence be generated by Algorithm 1 and suppose Assumptions 1-4 hold. Then, for and , the regret is bounded as follows:
[TABLE]
where
[TABLE]
Proof. Based on Algorithm 1 and double stochasticity of , we obtain that
[TABLE]
Thus, according to Assumptions 2 and 3, for any , we have that
[TABLE]
Recalling the regret notion defined in (2) and combining (III-B), we obtain that
[TABLE]
where the last inequality is obtained by using the fact .
Next, we aim to bound the term in (III-B). By using Assumption 4, we have
[TABLE]
where and the last inequality is obtained by using the fact [30]. It can be further verified that
[TABLE]
where the first inequality is obtained by utilizing the following optimality condition: and the last inequality is derived based on the convexity condition of together with Assumption 4. Then, it follows from (III-B) and (III-B) that
[TABLE]
where . Through simplifying (III-B) by using the method similar to Lemma 4 in [15], we obtain that
[TABLE]
For the first term on the right hand of (III-B), we obtain
[TABLE]
Through recalling the definition of and combining (III-B), (III-B), (23), (24), Lemmas 1 and 2, we can readily obtain the condition (1). The proof is complete.
In Theorem 1, (1) shows that the upper bound of the regret depends on and , where are the scalars consisting of the initial values, optimization problem parameters, and the network parameters . It should be pointed out that the shorter the jointed connection period of graph is, the tighter the regret bound will be due to the smaller parameter and in coefficients and . When the optimization problem and the network graph are determined, the coefficient will be a finite fixed constants and will have a fixed order over time . It is not hard to note that the regret bound of Algorithm 1 is affected by and . Thus, we have the following corollary.
Corollary 1
*Suppose that the conditions in Theorem 1 and hold. Given positive scalars and , the quantization resolution and the step size are chosen as , respectively. Then, we have that *
[TABLE]
where .
Proof. Substituting the conditions of in Corollary 1 into (1), once can verify that when , when , and when , respectively. Similarly, can be bounded as when , when , and when , respectively. Then, (1) is easily obtained based on different ranges of . The proof is complete.
Remark 4
Note that the setting of the decreasing quantization resolution in Corollary 1 allows relatively coarse quantization in the early stage of algorithm execution. In particular, when the parameter is chosen as a small value, the saving of communication resources will be significant but the regret bound will be poor, which implies that the setting of the parameter links the trade-off between them. It should be noted that when the total iteration time is large, the quantitative effect of information may be weakened to be close to the real value, especially in the later stage of algorithm operation. This change is actually reasonable since the state variables must approach final optima by continually obtaining the precise data as long as the algorithm runs.
Remark 5
The result of Corollary 1 matches the centralized result [31] and distributed results [15, 16] while taking quantized communication into account. Compared with [16], we additionally consider quantization communication and do not require the loss function to be bounded. Moreover, the requirements and in Corollary 1 imply that the cumulative variations of function value and gradient value grow slower than as increases. This also means that the loss functions and gradient functions satisfy certain regularities over time, such as the variability of function parameters decreases over time. According to Corollary 1, it is not hard to find that this requirement is reasonable and necessary for guaranteeing the sub-linearity of the considered dynamic regret. In addition, if the bound of the prior knowledge can be known in advance, i.e., , (1) can be improved to by setting .
IV Simulation
In this section, the following distributed online linear regression problem with a regularization term is simulated to verify the proposed algorithm.
[TABLE]
where , represents the feature and label information, and is a regular parameter. The feature vector is generated randomly and uniformly and its element satisfies . The label satisfies where is generated randomly in the interval . In the following simulation, we set the algorithm parameters , , , and take the probabilistic quantizer mentioned in Remark 1 as an example. To measure the performance of the algorithm, the global average dynamic regret is defined.
To investigate the convergence of Algorithm 1 and the effect of quantization parameters on algorithm convergence, we compare the global average dynamic regrets of Algorithm 1 under different cases: no quantization [15], quantization levels and . From Fig. 2, Algorithm 1 is convergent and when with a larger increasing tendency is selected, the related convergence performance is better. Note that in the early stage of the algorithm, the performance fluctuation caused by relatively coarse quantization resolution can be tolerated and this error can be weakened with the iteration time. Further, we analyze the effect of the quantization level with the maximum number on the convergence performance of the designed algorithm. Taking quantization level as an example and considering the case of the comparison results are shown in Fig. 2. It can be seen that when , its convergence curve is close to that without the maximum number, while , the convergence performance is poor. It should be noted that the design of quantization level with an appropriate parameter can better save communication resources than that without the limited parameter , but it always has a quantization error because the quantized data cannot approach the real-value data.
Next, we carry out a comparative study for the convergence performance of Algorithm 1 under the step size design taking into account unknown total iteration time as well as the quantization level . Without loss of generality, setting the step size as , respectively, the comparison results of the convergence performance are revealed in Fig. 3. Among the settings of step sizes, the dynamic regret under the step size with the knowledge of has a significantly better convergence effect, while that under the step size without the knowledge of has a large fluctuation of the convergence performance for different settings. Although the latter does not require prior knowledge of , it always has a performance gap in a theoretical sense according to Theorem 1. In addition, as the horizon varies, this step size without the horizon may cause the original convergence performance to be unmaintainable due to its invariant setting. Finally, the effect of the number of agents on the convergence performance is studied under . Through setting , the comparison of the global average dynamic regret is shown in Fig. 4, which verifies the theoretical results in Theorem 1 that the smaller the value of is, the better the convergence performance of Algorithm 1 is.
V CONCLUSIONS
For the distributed online constrained optimization problem under quantized communication, this paper has developed a quantized distributed online projection-free optimization algorithm. The use of random quantizers and linear oracle in the proposed algorithm has ensured the effective saving of communication resources and computational costs. For different settings of quantization resolution , the related dynamic regret bound has been established, in which the optimal bound can be achieved when the knowledge is known and . In addition, we have revealed the trade-off between the convergence performance and the quantization effect. Finally, a simulation example has been investigated to verify the theoretical results. A promising direction in the future is to investigate the nonconvex loss function case that will be more general yet more challenging.
-A Proof of lemma 1
According to Algorithm 1, we get
[TABLE]
According to Algorithm 1, the term can be further simplified as follows:
[TABLE]
where the second equality combines the double stochasticity of weight matrix .
Combining (-A) and (-A), for , we achieve
[TABLE]
where the second inequality follows the fact and the property [27] : , , .
Summing from to and to on both sides of (-A), we get
[TABLE]
where the second inequality follows the fact
[TABLE]
The proof is complete.
-B Proof of lemma 2
According to Assumption 4, we establish that
[TABLE]
For the first term on the right hand side of (-B), from Algorithm 1, for any it can be verified that
[TABLE]
Note that from Algorithm 1. Hence, the equality holds when . Now we assume that holds at time , and we intend to show the same conclusion at time . Actually,
[TABLE]
where the last equality follows from the double stochasticity of . With this condition, we obtain for any that
[TABLE]
Similar to (-A), combining the fact , it follows from (-B) and (-B) that
[TABLE]
This implies that
[TABLE]
For the second term on the right hand side of (-B), by recalling the notion defined in (12), it can be yielded that
[TABLE]
where the last inequality is obtained based on the fact:
[TABLE]
Substituting the above inequalities into (-B) and combining (-B) and fact , we can readily obtain (14). The proof is complete.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] X. Yi, X. Li, T. Yang, L. Xie, T. Chai, and K. H. Johansson, “Distributed bandit online convex optimization with time-varying coupled inequality constraints,” IEEE Transactions on Automatic Control , vol. 66, no. 10, pp. 4620–4635, 2021.
- 2[2] S. Shahrampour and A. Jadbabaie, “Distributed online optimization in dynamic environments using mirror descent,” IEEE Transactions on Automatic Control , vol. 63, no. 3, pp. 714–725, 2017.
- 3[3] D. Yuan, B. Zhang, D. W. Ho, W. X. Zheng, and S. Xu, “Distributed online bandit optimization under random quantization,” Automatica , vol. 146, p. 110590, 2022.
- 4[4] T. Yang, X. Yi, J. Wu, Y. Yuan, D. Wu, Z. Meng, Y. Hong, H. Wang, Z. Lin, and K. H. Johansson, “A survey of distributed optimization,” Annual Reviews in Control , vol. 47, pp. 278–305, 2019.
- 5[5] X. Li, L. Xie, and N. Li, “A survey of decentralized online learning,” ar Xiv preprint , vol. ar Xiv:2205.00473, 2022.
- 6[6] C. Liu, H. Li, Y. Shi, and D. Xu, “Distributed event-triggered gradient method for constrained convex minimization,” IEEE Transactions on Automatic Control , vol. 65, no. 2, pp. 778–785, 2020.
- 7[7] S. Sundhar Ram, A. Nedić, and V. V. Veeravalli, “Distributed stochastic subgradient projection algorithms for convex optimization,” Journal of Optimization Theory and Applications , vol. 147, no. 3, pp. 516–545, 2010.
- 8[8] F. Yan, S. Sundaram, S. Vishwanathan, and Y. Qi, “Distributed autonomous online learning: Regrets and intrinsic privacy-preserving properties,” IEEE Transactions on Knowledge and Data Engineering , vol. 25, no. 11, pp. 2483–2493, 2013.
