Approximating Tensor Norms via Sphere Covering: Bridging the Gap Between Primal and Dual

TL;DR

This paper introduces deterministic, data-independent algorithms for approximating tensor spectral and nuclear norms using sphere covering techniques, improving bounds and bridging the gap between primal and dual norms.

Contribution

It presents novel sphere covering constructions that enable the best approximation bounds for both tensor norms, enhancing efficiency and applicability.

Findings

Improved approximation bounds for tensor nuclear norm.

Deterministic, data-independent algorithms for tensor norms.

Sphere covering methods applicable to other decision problems.

Abstract

The matrix spectral and nuclear norms appear in enormous applications. The generalizations of these norms to higher-order tensors is becoming increasingly important but unfortunately they are NP-hard to compute or even approximate. Although the two norms are dual to each other, the best known approximation bound achieved by polynomial-time algorithms for the tensor nuclear norm is worse than that for the tensor spectral norm. In this paper, we bridge this gap by proposing deterministic algorithms with the best bound for both tensor norms. Our methods not only improve the approximation bound for the nuclear norm, but are also data independent and easily implementable comparing to existing approximation methods for the tensor spectral norm. The main idea is to construct a selection of unit vectors that can approximately represent the unit sphere, in other words, a collection of spherical caps to cover the sphere. For this purpose, we explicitly construct several collections of spherical caps for sphere covering with adjustable parameters for different levels of approximations and cardinalities. These readily available constructions are of independent interest as they provide a powerful tool for various decision making problems on spheres and related problems. We believe the ideas of constructions and the applications to approximate tensor norms can be useful to tackle optimization problems over other sets such as the binary hypercube.

Tables

Table 1. Table 1: Constructions of spherical caps to cover the unit sphere

Set of $𝒗$’s for ${𝔹}^n(𝒗,\tau )$	$\tau$ for ${𝔹}^n(𝒗,\tau )$	Number of ${𝔹}^n(𝒗,\tau )$’s
Any $\left\{𝒗\right\}$ where $𝒗\in {𝕊}^n$	$-1$	$1$
Any $\{𝒗,-𝒗\}$ where $𝒗\in {𝕊}^n$	$0$	$2$
Any regular simplex inscribed in ${𝕊}^n$	$1/n$	$n+1$
Any basis of ${ℝ}^n$ with their negations	$1/\sqrt{n}$	$2n$
${ℍ}_1^n$ (Section 2.1), ${ℍ}_4^n$ and ${ℍ}_5^n$ (Section 2.4)	$\Omega \left(\sqrt{\ln n/n}\right)$	$O\left(n^{\alpha }\right)$ for $\alpha >1$
${ℍ}_2^n$ (Section 2.2)	$\Omega \left(1/\sqrt{\ln n}\right)$	$O\left(3^n\right)$
${ℍ}_3^n$ (Section 2.3)	$\Omega \left(1\right)$	$O\left({\beta }^n\right)$ for $\beta >4$
Grid points in spherical coordinates	$1-O\left(n/m^2\right)$	$O\left(m^{n-1}\right)$

Table 2. Table 2: Properties of ℍ 3 n ​ ( α , α + 1 ) subscript superscript ℍ 𝑛 3 𝛼 𝛼 1 \mathbb{H}^{n}_{3}(\alpha,\alpha+1) for some α 𝛼 \alpha .

$\alpha$	17.42	7.64	4.75	$4.24$	4.00	3.00	2.00
$\tau$ for the $\tau$-hitting set	0.213	0.278	0.299	0.30028	0.300	0.288	0.235
$t$ for $\left|{ℍ}_3^n(\alpha ,\alpha +1)\right|\le t^n$	5.00	6.00	7.00	7.31	7.48	8.47	10.40

Table 3. Table 3: Exact and theoretical estimates of τ 𝜏 \tau and m 𝑚 m for hitting sets in 𝕊 6 superscript 𝕊 6 \mathbb{S}^{6}

Hitting set in ${𝕊}^6$	Exact $\tau$	Exact $m$	Estimated $\tau$	Estimated $m$
A regular simplex in ${𝕊}^6$	0.167	7	0.167	7
${𝔼}^6\cup \left(-{𝔼}^6\right)$	0.408	12	0.408	12
${ℍ}_1^6({\gamma }_1,{ϵ}_1)$	[0.331, 0.442]	27	${\omega }_1\cdot 0.546$	$o_1\cdot 6^{{\alpha }_1}$
${ℍ}_1^6({\gamma }_2,{ϵ}_2)$	[0.521, 0.592]	60	${\omega }_2\cdot 0.546$	$o_2\cdot 6^{{\alpha }_2}$
${ℍ}_4^6$	0.546	27	${\omega }_3\cdot 0.546$	$o_3\cdot 6^{2.792}$
${ℍ}_5^6(2+\sqrt{5},3+\sqrt{5})$	0.544	36	${\omega }_4\cdot 0.546$	$o_4\cdot 6^{2.989}$
${ℍ}_2^6$	0.835	728	${\omega }_5\cdot 0.747$	$3^6=729$
${ℍ}_3^6(2+\sqrt{5},3+\sqrt{5})$	0.820	16896	${\omega }_6\cdot 1.000$	${7.31}^6=152582$

Table 4. Table 4: Exact τ 𝜏 \tau and m 𝑚 m for Ω ​ ( ln ⁡ n / n ) Ω 𝑛 𝑛 \Omega\big{(}\sqrt{\ln n/n}\big{)} -hitting sets in 𝕊 n superscript 𝕊 𝑛 \mathbb{S}^{n}

	$n=6$		$n=8$		$n=12$		$n=15$
Hitting set	$\tau$	$m$	$\tau$	$m$	$\tau$	$m$	$\tau$	$m$
${ℍ}_1^n$	[0.331, 0.442]	24	[0.272, 0.384]	32	[0.368, 0.410]	104	[0.320, 0.391]	130
${ℍ}_1^n$	[0.521, 0.592]	60	[0.460, 0.502]	80	[0.441, 0.484]	184	[0.428, 0.452]	235
${ℍ}_4^n$	0.546	24	0.485	32	0.4653	104	0.431	130
${ℍ}_5^n$	0.544	36	0.489	48	0.4713	256	0.433	320

Table 5. Table 5: Numerical results for Examples 3.12 and 3.13 in  [ 28 ]

Example	Method	CPU	CPU+	Value	Value+	Bound	Bound+
Ex 3.12	[28]	0.703		2.8167		1.0000
	Alg 3.3	0.000	0.000	2.2076	2.8167	0.7837	1.0000
Ex 3.13	[28]	0.545	0.612	0.9862	1.0000	0.9862	1.0000
	Alg 3.3	0.000	0.250	0.8397	1.0000	0.8397	1.0000

Table 6. Table 6: Numerical results for Example 3.14 in  [ 28 ]

$n$	Method	CPU	CPU+	Value	Value+	Bound	Bound+	$\sqrt{\ln n/n}$
5	[28]	0.997		6.0996		1.0000
	Alg 3.3	0.020	0.050	4.3058	6.0996	0.7059	1.0000	0.5674
10	[28]	1.411		14.7902		1.0000
	Alg 3.3	0.320	1.920	8.4779	14.7902	0.5732	1.0000	0.4799
15	[28]	3.696		25.4829		1.0000
	Alg 3.3	1.670	3.680	11.4022	25.4829	0.4474	1.0000	0.4249
20	[28]	8.763		33.7020		1.0000
	Alg 3.3	4.870	20.120	13.3617	33.7020	0.3964	1.0000	0.3870
25	[28]	37.535		46.7997		1.0000
	Alg 3.3	50.310	110.000	19.5674	46.7997	0.4181	1.0000	0.3588
30	[28]	52.994		64.9106		1.0000
	Alg 3.3	101.380	152.160	24.5234	64.9106	0.3778	1.0000	0.3367
35	[28]	111.547		80.7697		1.0000
	Alg 3.3	197.510	350.360	28.6220	80.7697	0.3543	1.0000	0.3187
40	[28]	241.565		95.0878		1.0000
	Alg 3.3	362.230	548.350	33.7020	95.0878	0.3307	1.0000	0.3037

Table 7. Table 7: Approximating spectral norm by Algorithm 3.3 (using ℍ 1 n subscript superscript ℍ 𝑛 1 \mathbb{H}^{n}_{1} ) for n × 10 × 10 𝑛 10 10 n\times 10\times 10 tensors

$n$	5	10	20	30	40	50
$\sqrt{\ln n/n}$	0.5674	0.4799	0.3870	0.3367	0.3037	0.2797
Min bound	0.6317	0.6344	0.5751	0.5263	0.4663	0.4602
Min bound+	0.6921	0.6500	0.5847	0.5371	0.4768	0.8603
Max bound	0.9879	0.9611	0.8572	0.7653	0.7025	0.6579
Max bound+	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000
Mean bound	0.8898	0.8219	0.6786	0.6459	0.5778	0.5278
Mean bound+	0.9895	0.9896	0.9859	0.9825	0.9758	0.9932
% of optimality+	92.0%	92.0%	91.0%	87.5%	84.5%	89.0%
Mean CPU+	0.02	0.23	0.78	6.84	12.47	18.39

Table 8. Table 8: Approximating spectral norm by Algorithm 3.3 (using ℍ 1 10 subscript superscript ℍ 10 1 \mathbb{H}^{10}_{1} ) for 10 × n × n 10 𝑛 𝑛 10\times n\times n tensors

$n$	5	10	20	30	40	50
Min bound	0.6574	0.5016	0.5094	0.6858	0.5133	0.5905
Min bound+	0.6746	0.5321	0.5109	0.7236	0.5261	0.6099
Max bound	0.9451	0.9453	0.9472	0.9375	0.9819	0.9620
Max bound+	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000
Mean bound	0.8271	0.8292	0.8308	0.8280	0.8326	0.8295
Mean bound+	0.9899	0.9826	0.9893	0.9933	0.9814	0.9851
% of optimality+	90.0%	85.5%	91.0%	93.5%	88.5%	90.0%
Mean CPU+	0.06	0.23	0.76	1.81	3.12	5.53

Table 9. Table 9: Approximating nuclear norm by Algorithm 3.9 (using ℍ 1 n subscript superscript ℍ 𝑛 1 \mathbb{H}^{n}_{1} ) for n × 10 × 10 𝑛 10 10 n\times 10\times 10 tensors

$n$	5	10	20	30	40	50
$\sqrt{n/\ln n}$	1.7626	2.0840	2.5838	2.9699	3.2929	3.5751
Min bound	1.1791	1.2521	1.7417	1.8815	2.1672	2.5568
Max bound	1.4998	1.5263	2.0248	2.0187	2.2854	3.2108
Mean bound	1.3078	1.4135	1.9055	1.9522	2.2221	2.9763
Mean CPU+	0.69	13.31	101.49	1957.03	5365.99	11609.46

Table 10. Table 10: Approximating nuclear norm by Algorithm 3.9 (using ℍ 1 10 subscript superscript ℍ 10 1 \mathbb{H}^{10}_{1} ) for 10 × n × n 10 𝑛 𝑛 10\times n\times n tensors

$n$	5	10	20	30	40	50
Min bound	1.2999	1.3110	1.3008	1.3225	1.3638	1.3511
Max bound	1.5275	1.5303	1.5257	1.5405	1.5099	1.5726
Mean bound	1.4120	1.4112	1.4148	1.4148	1.4200	1.4239
Mean CPU+	2.06	13.23	160.03	941.46	6618.63	9488.85

Equations

$$\Vert \mathcal{T}{\Vert }_{\sigma }=\max \left\{\mathcal{T}(\mathbf{x},\mathbf{y},\mathbf{z}):\Vert \mathbf{x}{\Vert }_2=\Vert \mathbf{y}{\Vert }_2=\Vert \mathbf{z}{\Vert }_2=1,\mathbf{x},\mathbf{y},\mathbf{z}\in {\mathbb{R}}^n\right\},$$

$$\min \left\{\Vert \mathcal{T}-\lambda \mathbf{x}\otimes \mathbf{y}\otimes \mathbf{z}{\Vert }_{\text{F}}:\lambda \in \mathbb{R},\Vert \mathbf{x}{\Vert }_2=\Vert \mathbf{y}{\Vert }_2=\Vert \mathbf{z}{\Vert }_2=1,\mathbf{x},\mathbf{y},\mathbf{z}\in {\mathbb{R}}^n\right\},$$

$${\mathbb{B}}^n(\mathbf{v},\tau ):=\left\{\mathbf{x}\in {\mathbb{S}}^n:{\mathbf{x}}^{\text{T}}\mathbf{v}\ge \tau \right\}$$

$$\Vert \mathcal{T}{\Vert }_{\ast }=\min \left\{\sum _{i=1}^r|{\lambda }_i|:\mathcal{T}=\sum _{i=1}^r{\lambda }_i{\mathbf{x}}_i\otimes {\mathbf{y}}_i\otimes {\mathbf{z}}_i,{\lambda }_i\in \mathbb{R},\Vert {\mathbf{x}}_i{\Vert }_2=\Vert {\mathbf{y}}_i{\Vert }_2=\Vert {\mathbf{z}}_i{\Vert }_2=1,r\in \mathbb{N}\right\}.$$

$$\Vert \mathcal{T}{\Vert }_{\ast }=\underset{\Vert \mathcal{X}{\Vert }_{\sigma }\le 1}{\max }⟨\mathcal{T},\mathcal{X}⟩\text{mbox}and\Vert \mathcal{T}{\Vert }_{\sigma }=\underset{\Vert \mathcal{X}{\Vert }_{\ast }\le 1}{\max }⟨\mathcal{T},\mathcal{X}⟩,$$

$$⟨\mathcal{U},\mathcal{V}⟩:=\sum _{i_1=1}^{n_1}\sum _{i_2=1}^{n_2}\cdots \sum _{i_d=1}^{n_d}{u}_{i_1{i}_2\dots i_d}{v}_{i_1{i}_2\dots i_d}.$$

$$\mathbf{x}\otimes \mathbf{y}$$

$$\mathbf{x}⊠\mathbf{y}$$

$$\mathbf{x}\vee \mathbf{y}$$

$$\mathbb{T}(n,\tau,m):=\left\{\mathbb{H}^{n}\subseteq\mathbb{S}^{n}:\mathbb{H}^{n}\mbox{ is a $\tau$-hitting set},\,|\mathbb{H}^{n}|\leq m\right\}.$$

$$\begin{array}[]{ll}\mathbb{T}(n,\tau_{2},m)\subseteq\mathbb{T}(n,\tau_{1},m)&\mbox{if }\tau_{1}\leq\tau_{2}\\ \mathbb{T}(n,\tau,m_{1})\subseteq\mathbb{T}(n,\tau,m_{2})&\mbox{if }m_{1}\leq m_{2}.\end{array}$$

$$\{\mathbf{v}\}\in \mathbb{T}(n,-1,1)\text{mbox}and\{\mathbf{v},-\mathbf{v}\}\in \mathbb{T}(n,0,2)$$

$$\left\{{\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_{n+1}\right\}\in \mathbb{T}\left(n,\frac{1}{n},n+1\right).$$

$$\left\{\pm {\mathbf{v}}_1,\pm {\mathbf{v}}_2,\dots ,\pm {\mathbf{v}}_n\right\}\in \mathbb{T}\left(n,\frac{1}{\sqrt{n}},2n\right).$$

$$x_1$$

$$x_2$$

$$x_3$$

$$⋮$$

$$x_{n-1}$$

$$x_n$$

$${\mathbb{H}}_0^n(m):=\left\{\mathbf{x}\in {\mathbb{S}}^n:{\phi }_1,{\phi }_2,\dots ,{\phi }_{n-2}\in {\mathbb{D}}_1,{\phi }_{n-1}\in {\mathbb{D}}_2\right\}\in \mathbb{T}\left(n,1-\frac{{\pi }^2(n-1)}{8m^2},2m^{n-1}\right).$$

$$\Vert \mathbf{x}-\mathbf{z}\Vert \le \Vert \phi (\mathbf{x})-\phi (\mathbf{z})\Vert \le \frac{1}{2}\cdot \frac{\pi }{m}\cdot \sqrt{n-1}=\frac{\pi \sqrt{n-1}}{2m}.$$

$${\mathbf{x}}^{\text{T}}\mathbf{z}=\frac{1}{2}\left(2-\Vert \mathbf{x}-\mathbf{z}{\Vert }^2\right)\ge \frac{1}{2}\left(2-\frac{{\pi }^2(n-1)}{4m^2}\right)=1-\frac{{\pi }^2(n-1)}{8m^2}.$$

$$\text{Prob}\left\{{\mathbf{u}}^{\text{T}}\mathbf{v}\ge \sqrt{\frac{\gamma \ln n}{n}}\right\}\ge \frac{{\delta }_{\gamma }}{n^{2\gamma }\sqrt{\ln n}}.$$

$$\mathbb{H}^{n}_{1}(\gamma,\epsilon):=\left\{\boldsymbol{z}_{i}\mbox{ is i.i.d. uniform on $\mathbb{S}^{n}$ for }i=1,2,\dots,\left\lceil\kappa_{\gamma}n^{2\gamma}\sqrt{\ln n}\left(n\ln n+\ln\frac{1}{\epsilon}\right)\right\rceil\right\}$$

$$\text{Prob}\left\{{\mathbb{H}}_1^n(\gamma ,ϵ)\in \mathbb{T}\left(n,\sqrt{\frac{\gamma \ln n}{2n}},\lceil {\kappa }_{\gamma }{n}^{2\gamma }\sqrt{\ln n}\left(n\ln n+\ln \frac{1}{ϵ}\right)\rceil \right)\right\}\ge 1-ϵ.$$

$$\text{Prob}\left\{\mathbf{y}\notin \bigcup _{i=1}^t{\mathbb{B}}^n\left({\mathbf{z}}_i,\sqrt{\frac{\gamma \ln n}{n}}\right)\right\}=\text{Prob}\left\{\underset{1\le i\le t}{\max }{\mathbf{y}}^{\text{T}}{\mathbf{z}}_i<\sqrt{\frac{\gamma \ln n}{n}}\right\}\le {\left(1-\frac{{\delta }_{\gamma }}{n^{2\gamma }\sqrt{\ln n}}\right)}^t.$$

$$\text{Prob}\left\{{\mathbb{H}}_0^n(m)\subseteq \bigcup _{i=1}^t{\mathbb{B}}^n\left({\mathbf{z}}_i,\sqrt{\frac{\gamma \ln n}{n}}\right)\right\}\ge 1-2m^{n-1}{\left(1-\frac{{\delta }_{\gamma }}{n^{2\gamma }\sqrt{\ln n}}\right)}^t.$$

$$\text{Prob}\left\{\bigcup _{i=1}^t{\mathbb{B}}^n\left({\mathbf{z}}_i,\sqrt{\frac{\gamma \ln n}{2n}}\right)={\mathbb{S}}^n\right\}\ge \text{Prob}\left\{{\mathbb{H}}_0^n(m)\subseteq \bigcup _{i=1}^t{\mathbb{B}}^n\left({\mathbf{z}}_i,\sqrt{\frac{\gamma \ln n}{n}}\right)\right\}\ge 1-ϵ.$$

$${\mathbf{z}}^{\text{T}}\mathbf{x}\ge \cos ({\theta }_1+{\theta }_2)=\cos {\theta }_1\cos {\theta }_2-\sin {\theta }_1\sin {\theta }_2\ge \left(1-\frac{3}{2m}\right)\cdot \sqrt{\frac{\gamma \ln n}{n}}-\sqrt{\frac{3}{m}}\cdot 1\ge \sqrt{\frac{\gamma \ln n}{2n}}$$

$${\kappa }_{\gamma }{n}^{2\gamma }\sqrt{\ln n}\left(n\ln n+\ln \frac{1}{ϵ}\right)=\frac{\ln m}{\ln n}\cdot \frac{n^{2\gamma }\sqrt{\ln n}}{{\delta }_{\gamma }}\left(n\ln n+\ln \frac{1}{ϵ}\right)\ge \frac{n^{2\gamma }\sqrt{\ln n}}{{\delta }_{\gamma }}\left(n\ln m+\ln \frac{1}{ϵ}\right).$$

$${\mathbb{H}}_2^n:=\left\{\frac{\mathbf{z}}{\Vert \mathbf{z}\Vert }\in {\mathbb{S}}^n:\mathbf{z}\in \{-1,0,1{\}}^n,\Vert \mathbf{z}\Vert \ne 0\right\}.$$

$${\mathbb{Y}}_{\mathbb{D}}^n:=\left\{\mathbf{y}\in {\mathbb{R}}^n:y_i\in \{-1,1\}\text{mbox}fori\in \mathbb{D}\text{mbox}and{y}_i=0\text{mbox}fori\in \mathbb{I}\setminus \mathbb{D}\right\},$$

$$\underset{\mathbf{x}\in {\mathbb{S}}^n}{\min }\underset{\mathbf{z}\in {\mathbb{H}}_2^n}{\max }{\mathbf{x}}^{\text{T}}\mathbf{z}=\underset{\mathbf{x}\in {\mathbb{S}}^n}{\min }\underset{\mathbf{y}\in {\mathbb{Y}}^n}{\max }\frac{{\mathbf{x}}^{\text{T}}\mathbf{y}}{\Vert \mathbf{y}\Vert }=\underset{\mathbf{x}\in {\mathbb{S}}^n}{\min }\underset{\mathbb{D}\in 2^{\mathbb{I}}\setminus \{\varnothing \}}{\max }\underset{\mathbf{y}\in {\mathbb{Y}}_{\mathbb{D}}^n}{\max }\frac{{\mathbf{x}}^{\text{T}}\mathbf{y}}{\Vert \mathbf{y}\Vert }=\underset{\mathbf{x}\in {\mathbb{S}}^n}{\min }\underset{\mathbb{D}\in 2^{\mathbb{I}}\setminus \{\varnothing \}}{\max }\sum _{i\in \mathbb{D}}\frac{|x_i|}{\sqrt{|\mathbb{D}|}}\ge \frac{2}{\sqrt{\ln n+5}}.$$

Full text

Keywords: spectral norm, nuclear norm, sphere covering, spherical caps, polynomial optimization, approximation algorithm, approximation bound

Mathematics Subject Classification: 15A60, 52C17, 90C59, 68Q17

1 Introduction

With the advances in data collection and storage capabilities, massive multidimensional and multiway tensor data are being generated in a wide range of emerging applications [21]. Tensor computations and optimizations have been an active research area in the recent decade. Computing tensor norms are evidently essential in modelling various tensor optimization problems. One typical example is tensor completion (see e.g., [38]) in which the tensor nuclear norm is commonly used as the convex surrogate of the tensor rank. However, most tensor norms are NP-hard to compute [14], such as the spectral norm [13] and the nuclear norm [10] when the order of a tensor is more than two, a sharp contrast to matrices (tensors of order two) whose spectral and nuclear norms are easy to compute, e.g., using singular value decompositions.

The tensor spectral norm [24] is commonly known as the maximization of a multilinear form over Cartesian products of unit spheres, a standard higher-order generalization of the matrix spectral norm. Taking a tensor $\mathcal{T}=(t_{ijk})\in\mathbb{R}^{n\times n\times n}$ of order three as an example, its spectral norm

[TABLE]

where $\mathcal{T}(\boldsymbol{x},\boldsymbol{y},\boldsymbol{z})=\sum_{i=1}^{n}\sum_{j=1}^{n}\sum_{k=1}^{n}t_{ijk}x_{i}y_{j}z_{k}$ is a trilinear form of $(\boldsymbol{x},\boldsymbol{y},\boldsymbol{z})$. This is equivalent to the best rank-one approximation of the tensor $\mathcal{T}$ in the tensor community

[TABLE]

where $\|\bullet\|_{\textnormal{F}}$ stands for the Frobenius norm and $\otimes$ stands for the vector outer product, meaning that $\boldsymbol{x}\otimes\boldsymbol{y}\otimes\boldsymbol{z}$ is a rank-one tensor.

Although the tensor spectral norm is NP-hard to compute, it is easy to obtain feasible solutions of (1.1) to approximate this norm. There have been a lot of research works [34, 39, 23, 12, 16] on approximation algorithms of (1.1) in the optimization community since the seminal work of He et al. [13]. The best known worst-case bound to approximate (1.1) in polynomial time is $\Omega\left({\sqrt{\frac{\ln n}{n}}}\right)$ [34, 12]. One simple approach for this bound is a naive randomized algorithm in [12]:

Sample a vector $\boldsymbol{v}$ uniformly on the sphere111The $n$ in $\mathbb{S}^{n}$ refers to the dimension of the space in which this sphere of dimension $n-1$ lives. $\mathbb{S}^{n}:=\{\boldsymbol{x}\in\mathbb{R}^{n}:\|\boldsymbol{x}\|_{2}=1\}$ and compute the spectral norm of the resulting matrix, i.e., $\max_{\|\boldsymbol{x}\|_{2}=\|\boldsymbol{y}\|_{2}=1}\mathcal{T}(\boldsymbol{x},\boldsymbol{y},\boldsymbol{v})$; 2. 2.

Repeat the above procedure independently until the largest objective value from all samples hits the desired bound.

If we were able to sample all vectors in the unit sphere for $\boldsymbol{z}$, then this approach certainly finds $\max_{\|\boldsymbol{x}\|_{2}=\|\boldsymbol{y}\|_{2}=\|\boldsymbol{z}\|_{2}=1}\mathcal{T}(\boldsymbol{x},\boldsymbol{y},\boldsymbol{z})$. It is obviously not possible to cover the unit sphere by enumerating unit vectors. However, if we are allowed some tolerance, say an approximation ratio $\tau\in(0,1]$, then a sample unit vector $\boldsymbol{v}$ becomes a spherical cap

[TABLE]

with the angular radius $\theta=\arccos\tau$. In this setting, $\boldsymbol{v}$ is able to generate a $\tau$-approximate solution if and only if the spherical cap $\mathbb{B}^{n}(\boldsymbol{v},\tau)$ includes an optimal $\boldsymbol{z}$ in (1.1). Alternatively, if we have a collection of sample unit vectors whose corresponding spherical caps joining together covers the whole sphere, then the best one in this collection can generate a $\tau$-approximate solution. In fact, the above algorithm does imply a randomized cover of the sphere whose covering volume is at least $1-\epsilon$ for any $\epsilon>0$ with high probability. However, this is much weaker than what we need here and even cannot guarantee the existence of a full cover. One of the major contributions in this paper is to find a reasonable number of spherical caps to cover the sphere, deterministically and explicitly.

There are certainly lots of decision problems over spheres. Among them many are hard problems that approximate solutions are commonly acceptable such as wireless communications [36] and spherical facility location [37]. There are even harder problems where sphere covering seems irrelevant but it can be indeed helpful. One of these problems is computing the tensor nuclear norm. Taking $\mathcal{T}\in\mathbb{R}^{n\times n\times n}$ again as an example, its nuclear norm is

[TABLE]

The decomposition of $\mathcal{T}$ into rank-one tensors in (1.2) is known as a CANDECOMP/PARAFAC (CP) decomposition [15]. While CP decompositions usually require the number of rank-one terms to be minimum, there is no such constraint in (1.2). In fact, the tensor nuclear norm and spectral norm are dual to each other (see e.g., [25]), i.e.,

[TABLE]

where $\langle,\rangle$ stands for the Frobenius inner product. Computing or approximating tensor nuclear norm is much harder no matter using the definition (1.2) or the dual formulation—the corresponding feasibility problem is not easy at all. The situation is different for the tensor spectral norm as the feasibility to (1.1) is trivial. There are various methods [7, 31, 21, 4, 35, 28, 19, 6] to compute the tensor spectral norm in practice but there is only one known method [27] to compute the tensor nuclear norm, to the best of our knowledge. This crucial fact has resulted alternative concepts for the tensor nuclear norm in practice, such as the average nuclear norms of the matrix flattenings from three different ways. In terms of approximating the tensor nuclear norm, the best polynomial-time worst-case approximation bound is $\Omega\left(\frac{1}{\sqrt{n}}\right)$ via matrix flattenings [18] or partitions into matrix slices [22]. This bound is worse than the best known one $\Omega\left({\sqrt{\frac{\ln n}{n}}}\right)$ for the tensor spectral norm. It is natural to expect achieving this bound for the dual norm to the tensor spectral norm. As another major work in this paper, via certain reformulation and convex optimization proposed in [17], we are able to bridge the gap between the primal and dual norms, with the help of constructions of spherical caps for sphere covering.

Covering a sphere by identical spherical caps has been studied in computational geometry since the pioneering work of Rogers [32]. Instead of describing spherical caps via the angular radius, the caps are measured in normalized volume in the study. Specifically, by defining the normalized volume of a spherical cap to be its true volume over the volume of $\mathbb{S}^{n}$ (in this sense the normalized volume of $\mathbb{S}^{n}$ is one), sphere covering asks for a given positive integer $m$, what is the smallest $\delta$ such that there are $m$ spherical caps with normalized volume $\delta$ covering $\mathbb{S}^{n}$? The quantity $\delta m$ is called the density of the covering. Studying the bounds of this density has been the main research topic along this line. An upper bound of $O\left(n\ln n\right)$ for the covering density was obtained by Rogers [33] for sufficiently small $\delta$. This remains the best known asymptotic upper bound although there were improvements made in terms of the constant of the asymptotic bound and for any $\delta$ in [2, 8].

For the lower bound of covering density, there is not a clear answer in general other than the trivial one, i.e., $\delta m\geq 1$. Rogers [32] stated that the density of a covering cannot beat a natural strategy based on tiling $\mathbb{R}^{n}$ with regular simplices, known as the simplex bound, whose value remains a conjecture and unproven. Rogers [32] computed that for $\delta\rightarrow 0$ the density is close to $\frac{n}{e\sqrt{e}}$. It is believed that the density is $\Omega(n)$. Several special cases for the simplex bound have been confirmed, either for very small $\delta$ or for $\delta$ in large cap regime; see [20] and references therein. Other than the two trivial cases for $m=1$ and $m=2$ which correspond to $\delta=1$ and $\delta=\frac{1}{2}$, respectively, perhaps the first nontrivial work along this line is due to Lusternik and Schnirelmann [1]: If $n$ open or closed sets cover $\mathbb{S}^{n}$, then one of these contains a pair of antipodal points. This implies that if $\delta<\frac{1}{2}$ then $m\geq n+1$. An obvious lower bound of $\Omega(n)$ for any universal constant $\delta<\frac{1}{2}$.

There are two optimization problems that are relevant to sphere covering in the literature. The sphere coverage verification is to decide whether a given set of spherical caps cover the sphere or not. Petković et a. [29] showed that sphere coverage verification is NP-hard and proposed a recursive algorithm based on quadratic optimization. The spherical discrepancy is to find the furthest point in $\mathbb{S}^{n}$ to a given set of points $\{\boldsymbol{v}_{1},\boldsymbol{v}_{2},\dots,\boldsymbol{v}_{m}\}\subseteq\mathbb{S}^{n}$, i.e., $\min_{\boldsymbol{x}\in\mathbb{S}^{n}}\max_{1\leq i\leq m}\boldsymbol{x}^{\textnormal{T}}\boldsymbol{v}_{i}$. The spherical discrepancy is also NP-hard since it is the optimization version of sphere coverage verification who is a decision version of spherical discrepancy. Jones and McPartlon [20] proposed a multiplicative weights-based algorithm that obtains an approximation bound up to lower order terms.

Although there is extensive research on the density of sphere covering and related problems, they do not exactly serve the purpose of our study in this paper. The asymptotic bounds on the normalized volumes are not aligned with the goal to obtain approximation bounds based on inner products between unit vectors. The upper bounds obtained in [2] are existence results via a randomized approach. The construction in [30] works only in the large cap regime for $\delta=e^{-\sqrt{n}}$ which resulted the number of caps to be exponential in $n$. A recent work on spherical discrepancy minimization [20] showed an algorithm to generate spherical caps sequentially until a covering is satisfied but the running time to generate a cap is $O(n^{10})$. Our goal is to achieve a good balance between the approximation measured by $\cos\theta$ for the angular radius $\theta$ and the number of caps that are not too large, say bounded by a polynomial function of $n$. More importantly, we hope to obtain explicit constructions of spherical caps to cover the unit sphere. These will be of great beneficial to the algorithm and optimization community apart from our applications in approximating tensor norms. The products of our simple and explicit constructions, together with some trivial and known constructions, are summarized in Table 1.

This paper is organized as follows. After introducing some uniform notations, we propose various constructions of spherical caps for sphere covering and bound the ratio $\tau$ and number of caps of each construction in Section 2. We work around a key ratio $\Omega\left(\sqrt{\frac{\ln n}{n}}\right)$ which is the largest possible if the number of spherical caps is $O(n^{\alpha})$ for some universal constant $\alpha>1$, from randomization (Section 2.1) to deterministic covering (Section 2.4) with some interesting byproducts (Sections 2.2 and 2.3). In Section 3, we apply the covering results to approximate tensor norms. Specifically, we propose the first implementable and deterministic algorithm with the known best approximation bound for the tensor spectral norm and related polynomial optimization problems in Section 3.1. A deterministic algorithm with an improved approximation bound for the tensor nuclear norm is proposed in Section 3.2. Numerical performance of the proposed algorithms are reported in Section 3.3. Finally, some concluding remarks are given in Section 4.

Some uniform notations

Throughout this paper we uniformly adopt lowercase letters (e.g., $x$), boldface lowercase letters (e.g., $\boldsymbol{x}=\left(x_{i}\right)$), capital letters (e.g., $X=\left(x_{ij}\right)$), and calligraphic letters (e.g., $\mathcal{X}=\left(x_{i_{1}i_{2}\dots i_{d}}\right)$) to denote scalars, vectors, matrices, and higher-order (order three or more) tensors, respectively. Denote $\mathbb{R}^{n_{1}\times n_{2}\times\dots\times n_{d}}$ to be the space of real tensors of order $d$ with dimension $n_{1}\times n_{2}\times\dots\times n_{d}$. The same notation applies for a vector space and a matrix space when $d=1$ and $d=2$, respectively. Denote $\mathbb{N}$ to be the set of positive integers.

The Frobenius inner product between two tensors $\mathcal{U},\mathcal{V}\in\mathbb{R}^{n_{1}\times n_{2}\times\dots\times n_{d}}$ is defined as

[TABLE]

Its induced Frobenius norm is naturally defined as $\|\mathcal{T}\|:=\sqrt{\langle\mathcal{T},\mathcal{T}\rangle}$. The two terms automatically apply to tensors of order two (matrices) and tensors of order one (vectors) as well. This is the conventional norm (a norm without a subscript) used throughout the paper.

All blackboard bold capital letters denote sets, such as $\mathbb{R}^{n}$, the unit sphere $\mathbb{S}^{n}$, a spherical cap $\mathbb{B}^{n}(\boldsymbol{v},\tau)$, the standard basis $\mathbb{E}^{n}:=\{\boldsymbol{e}_{1},\boldsymbol{e}_{2},\dots,\boldsymbol{e}_{n}\}$ of $\mathbb{R}^{n}$, where the superscript $n$ indicates that the concerned set is a subset of $\mathbb{R}^{n}$. Three vector operations are used, namely the outer product $\otimes$, the Kronecker product $\boxtimes$, and appending vectors $\vee$. Specifically, if $\boldsymbol{x}\in\mathbb{R}^{n_{1}}$ and $\boldsymbol{y}\in\mathbb{R}^{n_{2}}$, then

[TABLE]

These three operators also apply to vector sets via element-wise operations.

As a convention, the notion $\Omega(f(n))$ means that there are positive universal constants $\alpha,\beta$ and $n_{0}$ such that $\alpha f(n)\leq\Omega(f(n))\leq\beta f(n)$ for all $n\geq n_{0}$, i.e., the same order of magnitude to $f(n)$.

2 Sphere covering by spherical caps

This section is devoted to explicit constructions of spherical caps to cover $\mathbb{S}^{n}$ in $\mathbb{R}^{n}$ for $n\geq 2$. Although this is more commonly denoted by $\mathbb{S}^{n-1}$ in the literature, our notation is to emphasize that the sphere resides in the space of $\mathbb{R}^{n}$ and to better understand the constructions via Kronecker products.

Recall that for $\boldsymbol{v}\in\mathbb{S}^{n}$ and $-1\leq\tau\leq 1$, $\mathbb{B}^{n}(\boldsymbol{v},\tau)=\left\{\boldsymbol{x}\in\mathbb{S}^{n}:\boldsymbol{x}^{\textnormal{T}}\boldsymbol{v}\geq\tau\right\}$ is a closed spherical cap with the angular radius $\arccos\tau$. Obviously, $\mathbb{B}^{n}(\boldsymbol{v},-1)=\mathbb{S}^{n}$, $\mathbb{B}^{n}(\boldsymbol{v},0)$ is a hemisphere, and $\mathbb{B}^{n}(\boldsymbol{v},1)$ is a single point. A set of unit vectors $\mathbb{H}^{n}=\{\boldsymbol{v}_{i}\in\mathbb{S}^{n}:i=1,2,\dots,m\}$ is called a $\tau$-hitting set with cardinality $m$ if $\bigcup_{i=1}^{m}\mathbb{B}^{n}\left(\boldsymbol{v}_{i},\tau\right)=\mathbb{S}^{n}$, i.e., the $m$ spherical caps cover the unit sphere. Denote all $\tau$-hitting sets of $\mathbb{S}^{n}$ with cardinality no more than $m$ to be

[TABLE]

It is easy to see the monotonicity, i.e.,

[TABLE]

We will be working around $\tau$-hitting sets with $\tau=\Omega\left(\sqrt{\frac{\ln n}{n}}\right)$. This is the largest possible if the cardinality of the hitting set is bounded by $O(n^{\alpha})$ with some universal constant $\alpha>1$; see e.g. [12]. Other useful $\tau$-hitting sets with larger $\tau$’s are also constructed as byproducts that are of independent interest. The aim is to construct hitting sets with the cardinality as small as possible. Let us first look at some elementary ones.

It is obvious that for any $\boldsymbol{v}\in\mathbb{S}^{n}$,

[TABLE]

both attaining the minimum cardinality. For $\tau>0$, the famous Lusternik-Schnirelmann theorem [1] rules out any possible $\tau$-hitting set with cardinality no more than $n$. There is an elegant construction of $\frac{1}{n}$-hitting sets with cardinality $n+1$. If $\boldsymbol{v}_{1},\boldsymbol{v}_{2},\dots,\boldsymbol{v}_{n+1}$ are the vertices of a regular simplex centered at the origin and inscribed in $\mathbb{S}^{n}$, then

[TABLE]

Detailed construction is easier to be obtained from $\mathbb{R}^{n+1}$ and is left to interested readers. Raising $\tau$ to $\frac{1}{\sqrt{n}}$ without increasing the number of vectors too much, one has for any basis $\{\boldsymbol{v}_{1},\boldsymbol{v}_{2},\dots,\boldsymbol{v}_{n}\}$ of $\mathbb{R}^{n}$,

[TABLE]

However, slightly increasing this threshold, say to $\sqrt{\frac{\ln n}{n}}$, will significantly increase the cardinality of a hitting set. As mentioned earlier, if the cardinality is bounded by a polynomial function of $n$, then the largest possible $\tau=\Omega\left(\sqrt{\frac{\ln n}{n}}\right)$.

Toward the extreme case that $\tau$ is close to one, the longitude and latitude of the Earth provide a clue. For any $\boldsymbol{x}=(x_{1},x_{2},\dots,x_{n})^{\textnormal{T}}\in\mathbb{S}^{n}$, we denote its spherical coordinates to be $(\varphi_{1},\varphi_{2},\dots,\varphi_{n-1})$ with $\varphi_{1},\varphi_{2},\dots,\varphi_{n-2}\in[0,\pi]$ and $\varphi_{n-1}\in[0,2\pi)$ such that

[TABLE]

If we let $\mathbb{D}_{1}=\left\{\frac{k\pi}{m}:k=0,1,\dots,m-1\right\}$ and $\mathbb{D}_{2}=\left\{\frac{k\pi}{m}:k=0,1,\dots,2m-1\right\}$, then the grid points in spherical coordinates (see [17, Lemma 3.1]) are

[TABLE]

To see why $\mathbb{H}^{n}_{0}(m)$ is such a hitting set. For any $\boldsymbol{z}\in\mathbb{S}^{n}$ with spherical coordinates $\varphi(\boldsymbol{z})$, there must exist $\boldsymbol{x}\in\mathbb{H}^{n}_{0}(m)$ with spherical coordinates $\varphi(\boldsymbol{x})$, such that

[TABLE]

Since $\|\boldsymbol{x}\|=\|\boldsymbol{z}\|=1$, the above further leads to

[TABLE]

2.1 Randomized \Omega\big{(}\sqrt{\ln n/n}\big{)}-hitting sets

It is instructive to consider randomized hitting sets via the uniform distribution on $\mathbb{S}^{n}$. This is also important as it guarantees the existence of $\Omega\left(\sqrt{\frac{\ln n}{n}}\right)$-hitting sets. The following probability bound (see e.g., [12, 3]) provides an insight of such a hitting set.

Lemma 2.1

For any $\gamma\in(0,\frac{n}{\ln n})$, if $\boldsymbol{u}$ and $\boldsymbol{v}$ are drawn independently and uniformly on $\mathbb{S}^{n}$, then there is a constant $\delta_{\gamma}$ depending on $\gamma$ only, such that

[TABLE]

In fact, it is not difficult to cover $1-\epsilon$ of the volume of the unit sphere for any $\epsilon>0$ by applying Lemma 2.1 with the union bound; see [12]. However, this is a much weaker statement than Theorem 2.2 below. In particular, the event of covering $1-\epsilon$ of the volume of $\mathbb{S}^{n}$ for any given $\epsilon>0$ does not even guarantee the existence of a full cover, hence being weaker than the latter event. The following randomized hitting set has a cardinality $O(n^{\alpha})$ for some constant $\alpha>1$.

Theorem 2.2

For any $\epsilon>0$ and $\gamma\in(0,\frac{n}{\ln n})$, there is a constant $\kappa_{\gamma}>0$ depending on $\gamma$ only, such that

[TABLE]

satisfies

[TABLE]

Proof. The sphere covering is established in two steps, a spherical grid $\mathbb{H}^{n}_{0}$ to cover the whole sphere and the randomized hitting set $\mathbb{H}^{n}_{1}$ to cover the grid.

According to (2.1) one has $\mathbb{H}^{n}_{0}(m)\in\mathbb{T}\left(n,1-\frac{\pi^{2}(n-1)}{8m^{2}},2m^{n-1}\right)$. Let $m\geq n$. For any $\boldsymbol{x}\in\mathbb{S}^{n}$, there exists $\boldsymbol{y}\in\mathbb{H}^{n}_{0}(m)$ such that $\boldsymbol{x}^{\textnormal{T}}\boldsymbol{y}\geq 1-\frac{\pi^{2}(n-1)}{8m^{2}}$. By Lemma 2.1, for any $\boldsymbol{z}_{i}\in\mathbb{H}^{n}_{1}(\gamma,\epsilon)$, there exists an $\delta_{\gamma}$ depending on $\gamma$ and $\textnormal{Prob}\,\left\{\boldsymbol{y}^{\textnormal{T}}\boldsymbol{z}_{i}\geq\sqrt{\frac{\gamma\ln n}{n}}\right\}\geq\frac{\delta_{\gamma}}{n^{2\gamma}\sqrt{\ln n}}$, i.e., $\textnormal{Prob}\,\left\{\boldsymbol{y}^{\textnormal{T}}\boldsymbol{z}_{i}<\sqrt{\frac{\gamma\ln n}{n}}\right\}\leq 1-\frac{\delta_{\gamma}}{n^{2\gamma}\sqrt{\ln n}}$. Denote $t=|\mathbb{H}^{n}_{1}(\gamma,\epsilon)|$. By the independence of $\boldsymbol{z}_{i}$’s, we have

[TABLE]

Since $\left|\mathbb{H}^{n}_{0}(m)\right|=2m^{n-1}$ and the points of $\mathbb{H}^{n}_{0}(m)$ are fixed, the probability that $\bigcup_{i=1}^{t}\mathbb{B}^{n}\left(\boldsymbol{z}_{i},\sqrt{\frac{\gamma\ln n}{n}}\right)$ fails to cover at least one point of $\mathbb{H}^{n}_{0}(m)$ is no more than $2m^{n-1}\left(1-\frac{\delta_{\gamma}}{n^{2\gamma}\sqrt{\ln n}}\right)^{t}$. In other words,

[TABLE]

By noticing that $m\geq n\geq 2$, it is not difficulty to verify that if $t\geq\frac{n^{2\gamma}\sqrt{\ln n}}{\delta_{\gamma}}\left(n\ln m+\ln\frac{1}{\epsilon}\right)$, then the right hand side of the above is at least $1-\epsilon$.

To summarize, if $\mathbb{H}^{n}_{0}(m)\subseteq\bigcup_{i=1}^{t}\mathbb{B}^{n}\left(\boldsymbol{z}_{i},\sqrt{\frac{\gamma\ln n}{n}}\right)$, then for any $\boldsymbol{x}\in\mathbb{S}^{n}$, there exists $\boldsymbol{y}\in\mathbb{H}^{n}_{0}(m)$ such that $\boldsymbol{y}^{\textnormal{T}}\boldsymbol{x}\geq 1-\frac{\pi^{2}(n-1)}{8m^{2}}$ and further there exists $\boldsymbol{z}\in\mathbb{H}^{n}_{1}(\gamma,\epsilon)$ such that $\boldsymbol{z}^{\textnormal{T}}\boldsymbol{y}\geq\sqrt{\frac{\gamma\ln n}{n}}$. If we are able to verify $\boldsymbol{z}^{\textnormal{T}}\boldsymbol{x}\geq\sqrt{\frac{\gamma\ln n}{2n}}$, then we must have $\bigcup_{i=1}^{t}\mathbb{B}^{n}\left(\boldsymbol{z}_{i},\sqrt{\frac{\gamma\ln n}{2n}}\right)=\mathbb{S}^{n}$. This finally leads to

[TABLE]

In order to show that $\boldsymbol{z}^{\textnormal{T}}\boldsymbol{x}\geq\sqrt{\frac{\gamma\ln n}{2n}}$, we let $\theta_{1}=\arccos(\boldsymbol{y}^{\textnormal{T}}\boldsymbol{x})$ and $\theta_{2}=\arccos(\boldsymbol{z}^{\textnormal{T}}\boldsymbol{y})$. Since $\cos\theta_{1}\geq 1-\frac{\pi^{2}(n-1)}{8m^{2}}\geq 1-\frac{3}{2m}$, one has $|\sin\theta_{1}|\leq\sqrt{1-\left(1-\frac{3}{2m}\right)^{2}}\leq\sqrt{\frac{3}{m}}$. Therefore,

[TABLE]

if $n\geq n_{0}$ for some $n_{0}$ that depends on $\gamma$ only. By choosing $m=n$ in $\mathbb{H}^{n}_{0}(m)$ and $\kappa_{\gamma}=\frac{1}{\delta_{\gamma}}$ and we have the desired $t$ for $n\geq n_{0}$.

To finish the final piece for remaining $n\leq n_{0}$, we may enlarge $m$ in $\mathbb{H}^{n}_{0}(m)$ in order for (2.2) to hold. If we choose $\kappa_{\gamma}=\frac{\ln m}{\delta_{\gamma}\ln n}$ correspondingly, this will ensure

[TABLE]

The largest $\kappa_{\gamma}$ for these finite $n\leq n_{0}$ provides the final $\kappa_{\gamma}$ that depends only on $n_{0}$ who itself depends on $\gamma$ only. $\Box$

Theorem 2.2 not only provides a simple construction with varying $\gamma$ but also trivially implies the existence of hitting sets in $\mathbb{T}\left(n,\Omega\left(\sqrt{\frac{\ln n}{n}}\right),O(n^{\alpha})\right)$. Although $\mathbb{H}^{n}_{1}(\gamma,\epsilon)$ is a full sphere covering with probability $1-\epsilon$ for any small $\epsilon>0$, it cannot be used to derive deterministic algorithms and even in some scenarios the feasibility may be questioned as we will see in approximating the tensor nuclear norm in Section 3.2. Moreover, to verify whether $\mathbb{H}^{n}_{1}(\gamma,\epsilon)$ covers the sphere or not, the sphere coverage verification, is NP-hard [29]. Therefore, explicit and deterministic constructions of hitting sets in $\mathbb{T}\left(n,\Omega\left(\sqrt{\frac{\ln n}{n}}\right),O(n^{\alpha})\right)$ are important. To get this job done, let us first look at two types $\tau$-hitting sets with larger $\tau$.

2.2 An \Omega\big{(}1/{\sqrt{\ln n}}\big{)}-hitting set

As the hitting ratio $\tau$ goes beyond $\Omega\left(\sqrt{\frac{\ln n}{n}}\right)$, we have to give up the polynomiality of $n$. Let us consider

[TABLE]

It is obvious that $|\mathbb{H}^{n}_{2}|<3^{n}$. We need to work out how large $\tau$ is for this $\tau$-hitting set, essentially Theorem 2.7 below. Interestingly, some results in matroid theory will be used in the proof.

To begin with, let $\mathbb{I}:=\{1,2,\dots,n\}$ and its power set $2^{\mathbb{I}}:=\{\mathbb{D}:\mathbb{D}\subseteq\mathbb{I}\}$. For any $\mathbb{D}\in 2^{\mathbb{I}}$, define

[TABLE]

and denote $\mathbb{Y}^{n}=\bigcup_{\mathbb{D}\in 2^{\mathbb{I}}\setminus\{\emptyset\}}\mathbb{Y}^{n}_{\mathbb{D}}$. It is easy to see that $\mathbb{H}^{n}_{2}=\left\{\frac{\boldsymbol{y}}{\|\boldsymbol{y}\|}:\boldsymbol{y}\in\mathbb{Y}^{n}\right\}$. Our goal is to establish a lower bound of $\min_{\boldsymbol{x}\in\mathbb{S}^{n}}\max_{\boldsymbol{z}\in\mathbb{H}^{n}_{2}}\boldsymbol{x}^{\textnormal{T}}\boldsymbol{z}$.

Theorem 2.3

It holds that

[TABLE]

It is straightforward to verify all the equalities in (2.3). To show the inequality, let us consider the following optimization problem

[TABLE]

where $\alpha\geq 0$ is a given constant. This is equivalent to

[TABLE]

which is to maximize a strictly convex quadratic function over a polyhedron

[TABLE]

Therefore, the optimal solution of (2.4) must be obtained at some extreme points of $\mathbb{X}^{n}$. To compute the optimal value, we now characterize extreme optimal points of (2.4). We need the following two technical results for the preparation.

Lemma 2.4

$g:2^{\mathbb{I}}\to\mathbb{R}$* where $g(\mathbb{D})=\alpha\sqrt{|\mathbb{D}|}$ with $\alpha>0$, then $\mathbb{X}^{n}$ is a polymatroid with respect to the function $g$ and the index set $\mathbb{I}$.*

Proof. It suffices to show that $g$ is a rank function, i.e., normalized, nondecreasing and submodular. Obviously, $g(\emptyset)=0$ and g(\mathbb{D}_{1})={\color[rgb]{0,0,0}\alpha\sqrt{|\mathbb{D}_{1}|}\leq\alpha\sqrt{|\mathbb{D}_{2}|}}=g(\mathbb{D}_{2}) whenever $\mathbb{D}_{1}\subseteq\mathbb{D}_{2}\subseteq\mathbb{I}$. It remains to show the submodularity

[TABLE]

If we let $|\mathbb{D}_{1}|=a$, $|\mathbb{D}_{2}\setminus\mathbb{D}_{1}|=b$, and $|\mathbb{D}_{1}\cap\mathbb{D}_{2}|=c$, then the above inequality is equivalent to

[TABLE]

This is actually implied by

[TABLE]

$\Box$

The next result is well known regarding an optimal solution of maximizing a linear function over a polymatroid; see e.g., [9].

Lemma 2.5

Consider the linear program

[TABLE]

where $\boldsymbol{a}\in\mathbb{R}^{n}_{+}$ and $g$ is a rank function. Let $\pi=(\pi_{1},\pi_{2},\dots,\pi_{n})$ be a permutation of $\mathbb{I}$ with $a_{\pi_{1}}\geq a_{\pi_{2}}\geq\dots\geq a_{\pi_{n}}\geq 0$. An optimal solution $\boldsymbol{x}$ to the linear program can be obtained by letting

[TABLE]

We can now characterize extreme optimal points and upper bound the optimal value of (2.4).

Proposition 2.6

The optimal value of (2.4) is no more than $\left(\frac{\ln n+5}{4}\right)\alpha^{2}$.

Proof. Denote $\boldsymbol{z}$ to be an optimal solution of (2.4). In fact, $\boldsymbol{z}$ is the unique optimal solution to the linear program

[TABLE]

If this is not true, we then have another $\boldsymbol{y}\in\mathbb{X}^{n}$ with $\boldsymbol{y}\neq\boldsymbol{z}$ and ${\boldsymbol{z}}^{\textnormal{T}}\boldsymbol{y}\geq\boldsymbol{z}^{\textnormal{T}}\boldsymbol{z}=\|\boldsymbol{z}\|^{2}$. This implies that $\|\boldsymbol{y}\|^{2}>\|\boldsymbol{z}\|^{2}$, invalidating the optimality of $\boldsymbol{z}$ to (2.4).

Applying Lemma 2.4 and Lemma 2.5 to (2.5) with $g(\mathbb{D})=\alpha\sqrt{|\mathbb{D}|}$ and $\boldsymbol{a}=\boldsymbol{z}\in\mathbb{R}^{n}_{+}$ and choosing a permutation $\pi$ with $z_{\pi_{1}}\geq z_{\pi_{2}}\geq\dots\geq z_{\pi_{n}}\geq 0$, one has

[TABLE]

As a consequence,

[TABLE]

which shows that the optimal value of (2.4), $\|\boldsymbol{z}\|^{2}$, is upper bounded by $\left(\frac{\ln n+5}{4}\right)\alpha^{2}$. $\Box$

We are ready to finish the final piece, i.e., to show the inequality in (2.3). If this is not ture, then there is an $\boldsymbol{x}\in\mathbb{S}^{n}$ such that

[TABLE]

This means that $|\boldsymbol{x}|\in\mathbb{R}^{n}_{+}$ with $\|\boldsymbol{x}\|^{2}=1$ is a feasible solution to (2.4) for $\alpha=\frac{2\beta}{\sqrt{\ln n+5}}$. However, according to Proposition 2.6, the optimal value of this problem is no more than

[TABLE]

giving rise to a contradiction. Finally, we conclude this part as below.

Corollary 2.7

It holds that

[TABLE]

2.3 $\Omega(1)$-hitting sets

$\mathbb{H}^{n}_{2}$ is simple and almost close to help the construction of deterministic $\Omega\left(\sqrt{\frac{\ln n}{n}}\right)$-hitting sets, whose story will be revealed in the next subsection. To make this final small but important step, $\Omega(1)$-hitting sets are needed. A finely tuned version of $\mathbb{H}^{n}_{2}$ is in place.

Algorithm 2.8

Given $\mathbb{S}^{n}$ and two parameters $\alpha\geq 1$ and $\beta\geq\alpha+1$, construct $\mathbb{H}^{n}_{3}(\alpha,\beta)$.

Let $m=\left\lceil\log_{\beta}\alpha n\right\rceil$ and partition $\mathbb{I}$ into disjoint subsets $\mathbb{I}_{1},\mathbb{I}_{2},\dots,\mathbb{I}_{m}$ such that

[TABLE] 2. 2.

For any partition $\{\mathbb{I}_{1},\mathbb{I}_{2},\dots,\mathbb{I}_{m}\}$ of $\mathbb{I}$ satisfying (2.6), construct a set of vectors

[TABLE] 3. 3.

Put all $\mathbb{Z}^{n}(\mathbb{I}_{1},\mathbb{I}_{2},\dots,\mathbb{I}_{m})$’s together to form

[TABLE] 4. 4.

Project the vectors in $\mathbb{Z}^{n}$ onto the unit sphere, i.e., $\mathbb{H}^{n}_{3}(\alpha,\beta):=\left\{\frac{\boldsymbol{z}}{\|\boldsymbol{z}\|}\in\mathbb{S}^{n}:\boldsymbol{z}\in\mathbb{Z}^{n}\right\}.$

We see from the first step of Algorithm 2.8 that

[TABLE]

implying that

[TABLE]

Therefore, the feasibility of the construction is guaranteed. On the other hand, as $m=\left\lceil\log_{\beta}\alpha n\right\rceil$, we have

[TABLE]

implying that

[TABLE]

Theorem 2.9

*For any $\boldsymbol{x}\in\mathbb{S}^{n}$, there exists $\boldsymbol{z}\in\mathbb{H}^{n}_{3}(\alpha,\beta)$ such that $\boldsymbol{z}^{\textnormal{T}}\boldsymbol{x}\geq\frac{\alpha-1}{\sqrt{\alpha\beta(\alpha+1)}}$. *

Proof. For any given $\|\boldsymbol{x}\|=1$, define the index sets

[TABLE]

For any entry $x_{i}$ of $\boldsymbol{x}$, $|x_{i}|\leq 1\leq\sqrt{\frac{\beta^{m}}{\alpha n}}$, and so $\{\mathbb{D}_{0}(\boldsymbol{x}),\mathbb{D}_{1}(\boldsymbol{x}),\dots,\mathbb{D}_{m}(\boldsymbol{x})\}$ is a partition of $\mathbb{I}$.

We first estimate $|\mathbb{D}_{k}(\boldsymbol{x})|$ for $k\geq 2$. It is obvious that for $k\geq 2$,

[TABLE]

This implies that $|\mathbb{D}_{k}(\boldsymbol{x})|<\frac{\alpha n}{\beta^{k-1}}$, i.e., $|\mathbb{D}_{k}(\boldsymbol{x})|\leq\left\lfloor\frac{\alpha n}{\beta^{k-1}}\right\rfloor$ for $k=2,3,\dots,m$. Hence, there exists a partition $\{\mathbb{I}_{1},\mathbb{I}_{2},\dots,\mathbb{I}_{m}\}$ of $\mathbb{I}$ satisfying (2.6) such that $\mathbb{D}_{k}(\boldsymbol{x})\subseteq\mathbb{I}_{k}$ for $k=2,3,\dots,m$. Furthermore, we may find a vector $\boldsymbol{z}\in\mathbb{Z}^{n}(\mathbb{I}_{1},\mathbb{I}_{2},\dots,\mathbb{I}_{m})$ such that

[TABLE]

where the sign  function takes $1$ for nonnegative reals and $-1$ for negative reals.

In the following, we shall estimate $\boldsymbol{z}^{\textnormal{T}}\boldsymbol{x}$ and $\|\boldsymbol{z}\|$. First of all,

[TABLE]

Next, we have

[TABLE]

Summing the above two inequalities would give us $\|\boldsymbol{z}\|^{2}\leq(\alpha+1)n$ since $\{\mathbb{D}_{0}(\boldsymbol{x}),\mathbb{D}_{1}(\boldsymbol{x}),\dots,\mathbb{D}_{m}(\boldsymbol{x})\}$ is a partition of $\mathbb{I}$.

Lastly, as $\textnormal{sign}\,(x_{i})=\textnormal{sign}\,(z_{i})$ for every $i\in\mathbb{I}$,

[TABLE]

where the second inequality is due to the upper bound in (2.9).

To conclude, we find $\frac{\boldsymbol{z}}{\|\boldsymbol{z}\|}\in\mathbb{H}^{n}_{3}(\alpha,\beta)$ such that

[TABLE]

proving the desired inequality. $\Box$

We also need to estimate the cardinality of $\mathbb{H}^{n}_{3}(\alpha,\beta)$. To simplify the display, we now replace $\beta$ by $\gamma+1$ where $\gamma\geq\alpha$ in the rest of this subsection.

Proposition 2.10

Given two constants $1\leq\alpha\leq\gamma$, one has

[TABLE]

Proof. For any given partition $\{\mathbb{I}_{1},\mathbb{I}_{2},\dots,\mathbb{I}_{m}\}$ of $\mathbb{I}$ satisfying (2.6), $x_{i}$ can take 2 values if $i\in\mathbb{I}_{1}$ and 4 values if $i\in\mathbb{I}_{k}$ for $k\geq 2$. By considering the number of such partitions and possible overlaps after projecting on to $\mathbb{S}^{n}$, one has

[TABLE]

By (2.7), we have

[TABLE]

It remains to estimate $\frac{n!}{\prod_{k=1}^{m}|\mathbb{I}_{k}|!}$ based on the followings from (2.6), (2.7) and (2.8) with $\beta=\gamma+1$,

[TABLE]

where $\eta_{1}=1-\frac{\alpha}{\gamma}$ and $\eta_{k}=\frac{\alpha}{(\gamma+1)^{k-1}}$ for $k=2,3\dots,m$, as well as $\sum_{k=1}^{m}|\mathbb{I}_{k}|=n$, $m=\left\lceil\log_{\gamma+1}\alpha n\right\rceil$ and $1\leq\alpha\leq\gamma$.

We first notice that

[TABLE]

We then consider the function $f(x)=x\ln x-x$, which is increasing and convex over $[1,\infty)$ since $f^{\prime}(x)=\ln x$ and $f^{\prime\prime}(x)=\frac{1}{x}$. Therefore, for any $x\geq 1$ and $y>0$, $\frac{f(x+y)-f(x)}{y}\leq f^{\prime}(x+y)=\ln(x+y)$, implying that $f(x+y)-f(x)\leq y\ln(x+y)$. Applying this fact to $|\mathbb{I}_{k}|$ for $k=1,2,\dots,m$, we obtain

[TABLE]

These, together with the Stirling approximation $\ln(n!)=f(n)+O(\ln n)$, lead to

[TABLE]

We are ready to estimate $\frac{n!}{\prod_{k=1}^{m}|\mathbb{I}_{k}|!}$ within a deviation of $o(n)$ as follows:

[TABLE]

where the last equality is due to the fact that $m=\left\lceil\log_{\gamma+1}\alpha n\right\rceil$ implies

[TABLE]

Finally, by combining the upper bound of $2^{|\mathbb{I}_{1}|}\prod_{k=2}^{m}4^{|\mathbb{I}_{k}|}$, we have $|\mathbb{H}^{n}_{3}(\alpha,\gamma+1)|\leq t^{n}$ where

[TABLE]

$\Box$

In a nutshell, we have the following.

Corollary 2.11

For any given $1\leq\alpha\leq\gamma$,

[TABLE]

For a fixed $\alpha\geq 1$, the largest $\frac{\alpha-1}{\sqrt{\alpha(\alpha+1)(\gamma+1)}}$ is $\frac{\alpha-1}{(\alpha+1)\sqrt{\alpha}}$, achieved when $\gamma=\alpha$. Correspondingly,

[TABLE]

If we maximize $\frac{\alpha-1}{(\alpha+1)\sqrt{\alpha}}$ to achieve the best $\tau$ for the $\tau$-hitting set, this is $\sqrt{\frac{1}{2}(5\sqrt{5}-11)}\approx 0.30028$ when $\alpha=2+\sqrt{5}\approx 4.236$. For reference, we list the $\tau$ and the cardinality for some $\mathbb{H}^{n}_{3}(\alpha,\alpha+1)$ in Table 2.

If we are interested to minimize the upper bound of $|\mathbb{H}^{n}_{3}(\alpha,\gamma+1)|$ in (2.10), then by fixing $\alpha$ and choosing $\gamma$ sufficiently large, the bound can even be $(2+\epsilon)^{n}$. However, this makes sense only if $\gamma\ll n$. Moreover, the corresponding $\tau=\frac{\alpha-1}{\sqrt{\alpha(\alpha+1)(\gamma+1)}}$ will decrease quickly as $\gamma$ goes large.

2.4 \Omega\big{(}\sqrt{\ln n/n}\big{)}-hitting sets

The hitting sets in Sections 2.2 and 2.3 can be used to construct new hitting sets which in fact derandomize the constructions in Section 2.1. Recall that $\mathbb{E}^{n}=\{\boldsymbol{e}_{1},\boldsymbol{e}_{2},\dots,\boldsymbol{e}_{n}\}$ is the standard basis of $\mathbb{R}^{n}$, $\boxtimes$ denotes the Kronecker product, and $\vee$ denotes vector appending.

Lemma 2.12

If a hitting set $\mathbb{H}^{n_{1}}\in\mathbb{T}(n_{1},\tau,m)$ with $\tau\geq 0$, then $\mathbb{E}^{n_{2}}\boxtimes\mathbb{H}^{n_{1}}\in\mathbb{T}\left(n_{1}n_{2},\frac{\tau}{\sqrt{n_{2}}},mn_{2}\right)$.

Proof. First, for any $\boldsymbol{e}_{i}\in\mathbb{E}^{n_{2}}$ and $\boldsymbol{z}\in\mathbb{H}^{n_{1}}$, one has $\|\boldsymbol{e}_{i}\boxtimes\boldsymbol{z}\|=\|\boldsymbol{e}_{i}\|\cdot\|\boldsymbol{z}\|=1$. Thus, $\mathbb{E}^{n_{2}}\boxtimes\mathbb{H}^{n_{1}}\subseteq\mathbb{S}^{n_{1}n_{2}}$. For any $\boldsymbol{x}\in\mathbb{S}^{n_{1}n_{2}}$, let $\boldsymbol{x}=\boldsymbol{x}_{1}\vee\boldsymbol{x}_{2}\vee\dots\vee\boldsymbol{x}_{n_{2}}$ where $\boldsymbol{x}_{k}\in\mathbb{R}^{n_{1}}$ for $k=1,2,\dots,n_{2}$. Since $\sum_{k=1}^{n_{2}}\|\boldsymbol{x}_{k}\|^{2}=\|\boldsymbol{x}\|^{2}=1$, there exists an $\boldsymbol{x}_{i}$, such that $\|\boldsymbol{x}_{i}\|^{2}\geq\frac{1}{n_{2}}$.

Observing that $\frac{\boldsymbol{x}_{i}}{\|\boldsymbol{x}_{i}\|}\in\mathbb{S}^{n_{1}}$, there exists $\boldsymbol{y}\in\mathbb{H}^{n_{1}}$ such that $\boldsymbol{y}^{\textnormal{T}}\frac{\boldsymbol{x}_{i}}{\|\boldsymbol{x}_{i}\|}\geq\tau$. Therefore, we have $\boldsymbol{e}_{i}\boxtimes\boldsymbol{y}\in\mathbb{E}^{n_{2}}\boxtimes\mathbb{H}^{n_{1}}$ satisfying

[TABLE]

Finally, by noticing possible overlaps, one has $|\mathbb{E}^{n_{2}}\boxtimes\mathbb{H}^{n_{1}}|\leq|\mathbb{E}^{n_{2}}|\cdot|\mathbb{H}^{n_{1}}|\leq n_{2}m$. $\Box$

Lemma 2.13

If two hitting sets $\mathbb{H}^{n_{1}}\in\mathbb{T}(n_{1},\tau_{1},m_{1})$ and $\mathbb{H}^{n_{2}}\in\mathbb{T}(n_{2},\tau_{2},m_{2})$ with $\tau_{1},\tau_{2}>0$, then

[TABLE]

Proof. For any $\boldsymbol{x}\in\mathbb{S}^{n_{1}+n_{2}}$, let $\boldsymbol{x}=\boldsymbol{x}_{1}\vee\boldsymbol{x}_{2}$ where $\boldsymbol{x}_{1}\in\mathbb{R}^{n_{1}}$ and $\boldsymbol{x}_{2}\in\mathbb{R}^{n_{2}}$. If one of them is a zero vector, say $\boldsymbol{x}_{1}={\bf 0}_{n_{1}}$, then $\|\boldsymbol{x}_{2}\|=1$. There exists $\boldsymbol{y}\in\mathbb{H}^{n_{2}}$ such that $\boldsymbol{y}^{\textnormal{T}}\boldsymbol{x}_{2}\geq\tau_{2}$, and so

[TABLE]

If both $\boldsymbol{x}_{1}$ and $\boldsymbol{x}_{2}$ are nonzero, then $\frac{\boldsymbol{x}_{k}}{\|\boldsymbol{x}_{k}\|}\in\mathbb{S}^{n_{k}}$ for $k=1,2$. There exist $\boldsymbol{y}_{k}\in\mathbb{H}^{n_{k}}$ with $\frac{\boldsymbol{y}_{k}^{\textnormal{T}}\boldsymbol{x}_{k}}{\|\boldsymbol{x}_{k}\|}\geq\tau_{k}$ for $k=1,2$. We have

[TABLE]

As $\|\boldsymbol{x}_{1}\|^{2}+\|\boldsymbol{x}_{2}\|^{2}=\|\boldsymbol{x}\|^{2}=1$, we must have either $\|\boldsymbol{x}_{1}\|\geq\frac{\tau_{2}}{\sqrt{{\tau_{1}}^{2}+{\tau_{2}}^{2}}}$ or $\|\boldsymbol{x}_{2}\|\geq\frac{\tau_{1}}{\sqrt{{\tau_{1}}^{2}+{\tau_{2}}^{2}}}$. In any case, we have

[TABLE]

implying that either $\boldsymbol{y}_{1}\vee{\bf 0}_{n_{2}}$ or ${\bf 0}_{n_{1}}\vee\boldsymbol{y}_{2}$ is close enough to $\boldsymbol{x}$. $\Box$

We are ready to construct new hitting sets using $\mathbb{H}^{n}_{2}\in\mathbb{T}\left(n,\Omega\left(\frac{1}{\sqrt{\ln n}}\right),3^{n}\right)$ in Section 2.2 and $\mathbb{H}^{n}_{3}\in\mathbb{T}\left(n,\mu,\nu^{n}\right)$ with universal constants $\mu,\nu>0$, a handy notation of (2.11) in Section 2.3.

Theorem 2.14

Given integer $n\geq 2$, let $n_{1}=\lceil\ln n\rceil$, $n_{2}=\lfloor\frac{n}{n_{1}}\rfloor$, and $n_{3}=n-n_{1}n_{2}$. One has

[TABLE]

Proof. First, as $n_{3}=n-n_{1}\lfloor\frac{n}{n_{1}}\rfloor<n_{1}$, we have

[TABLE]

According to Corollary 2.7 and Lemma 2.12,

[TABLE]

Besides, one has

[TABLE]

Noticing that $\frac{1}{\sqrt{\ln\ln n}}\geq\sqrt{\frac{\ln n}{n\ln\ln n}}$, (2.12) can be obtained by applying Lemma 2.13. $\Box$

Although $\Omega\left(\sqrt{\frac{\ln n}{n\ln\ln n}}\right)$ is slightly lower than $\Omega\left(\sqrt{\frac{\ln n}{n}}\right)$, the construction of $\mathbb{H}^{n}_{4}$ in (2.12) is very simple (using $\mathbb{H}^{n}_{2}$) and enjoys a low cardinality $O(n^{1+\ln 3})\leq O(n^{2.1})$. We remark that it is even possible to construct an $\Omega\left(\sqrt{\frac{\ln n}{n\ln\ln n}}\right)$-hitting set with a lower cardinality $O(n^{1.5})$. This can be done by using $\mathbb{H}^{n_{1}}_{0}(m)$ with $m=\lceil\sqrt{\ln n}\rceil$ and $n_{1}=\lceil\frac{\ln n}{\ln\ln n}\rceil$ in place of $\mathbb{H}^{n_{1}}_{2}$ in constructing $\mathbb{H}^{n}_{4}$ in Theorem 2.14. We leave the details to interested readers. In order to remove the {\color[rgb]{0,0,0}\frac{1}{\sqrt{\ln\ln n}}} factor, we need to make use of $\mathbb{H}^{n}_{3}(\alpha,\beta)$.

Theorem 2.15

Given integer $n\geq 2$, let $n_{1}=\lceil\ln n\rceil$, $n_{2}=\lfloor\frac{n}{n_{1}}\rfloor$, and $n_{3}=n-n_{1}n_{2}$. By choosing any $\mathbb{H}^{n}_{3}(\alpha,\beta)\in\mathbb{T}\left(n,\mu,\nu^{n}\right)$ in (2.11) with $\alpha\geq 1$ and $\beta\geq\alpha+1$, one has

[TABLE]

Proof. The proof is similar to that of Theorem 2.14 by noticing (2.13) and applying Lemma 2.12 and Lemma 2.13. We only need to carry out the calculations.

The $\tau$ for the $\tau$-hitting set $\mathbb{E}^{n_{2}}\boxtimes\mathbb{H}^{n_{1}}_{3}(\alpha,\beta)$ is $\frac{\mu}{\sqrt{n_{2}}}$ and that for $\mathbb{H}^{n_{3}}_{3}(\alpha,\beta)$ is $\mu$. By Lemma 2.13, the $\tau$ for $\mathbb{H}^{n}_{5}(\alpha,\beta)$ is $\frac{\frac{\mu}{\sqrt{n_{2}}}\cdot\mu}{\sqrt{\frac{\mu^{2}}{n_{2}}+\mu^{2}}}=\frac{\mu}{\sqrt{n_{2}+1}}\geq\mu\sqrt{\frac{\ln n}{n+\ln n}}.$

For the cardinality, one has $|\mathbb{E}^{n_{2}}\boxtimes\mathbb{H}^{n_{1}}_{3}(\alpha,\beta)|\leq n_{2}\nu^{n_{1}}\leq\frac{\nu n^{1+\ln\nu}}{\ln n}$ and $|\mathbb{H}^{n_{3}}_{3}(\alpha,\beta)|\leq\nu^{n_{3}}\leq n^{\ln\nu}$. Adding up these two would give $O(n^{1+\ln\nu})$. $\Box$

With the cardinality $O(n^{1+\ln\nu})$ in place, it is natural to select the best $\mu$ for $\mathbb{H}^{n}_{3}(\alpha,\beta)$ with $\beta=\gamma+1$ in (2.11). According to Table 2, the largest $\mu=0.30028$ with $\nu=7.31$, obtained when $\alpha=2+\sqrt{5}$ and $\beta=3+\sqrt{5}$. This results a cardinality $O(n^{1+\ln\nu})\leq O(n^{3})$. To conclude, we have

[TABLE]

Before concluding this section, we remark that the estimated $\tau$ serves a lower bound and $m$ serves an upper bound for the proposed hitting sets. We evaluate their exact values of one example ($n=6$) by numerical computations, shown in Table 3 where the Greek letters in estimated values are some unknown constants. Due to the randomness of $\mathbb{H}^{n}_{1}$, we try ten times for any $m$ and provide corresponding $\tau$ by an interval range.

For our main constructions of $\Omega\left(\sqrt{\frac{\ln n}{n}}\right)$-hitting sets, $\mathbb{H}_{1}^{n}$, $\mathbb{H}_{4}^{n}$ and $\mathbb{H}_{5}^{n}$, comparisons of $\tau$ and $m$ for a few small $n$’s are shown in Table 4. We observe that random hitting sets will outperform deterministic ones when $n$ increases although they are worse when $n$ is small.

3 Approximating tensor norms

In this section we apply explicit constructions for sphere covering, in particular the deterministic $\Omega\left(\sqrt{\frac{\ln n}{n}}\right)$-hitting sets in Section 2.4, to derive new approximation methods for the tensor spectral norm and nuclear norm. Let us formally define the approximation bound for tensor norms.

Definition 3.1

A tensor norm $\|\bullet\|_{\omega}$ can be approximated with an approximation bound $\tau\in(0,1]$, if there exists a polynomial-time algorithm that computes a quantity $\omega_{\mathcal{T}}$ for any tensor instance $\mathcal{T}$ in the concerned space, such that $\tau\|\mathcal{T}\|_{\omega}\leq\omega_{\mathcal{T}}\leq\|\mathcal{T}\|_{\omega}$.

Obviously the larger the $\tau$, the better the approximation bound. We consider the tensor space $\mathbb{R}^{n_{1}\times n_{2}\times\dots\times n_{d}}$ of order $d\geq 3$ and assume without loss of generality that $2\leq n_{1}\leq n_{2}\leq\dots\leq n_{d}$.

3.1 Approximation bound for tensor spectral norm

Given a tensor $\mathcal{T}\in\mathbb{R}^{n_{1}\times n_{2}\times\dots\times n_{d}}$, let us denote (recall that $\otimes$ stands for the outer product)

[TABLE]

to be the multilinear function of vector entries $(\boldsymbol{x}_{1},\boldsymbol{x}_{2},\dots,\boldsymbol{x}_{d})$ where $\boldsymbol{x}_{k}\in\mathbb{R}^{n_{k}}$ for $k=1,2,\dots,d$. If any vector entry, say $\boldsymbol{x}_{1}$, is missing and replaced by $\bullet$, then $\mathcal{T}(\bullet,\boldsymbol{x}_{2},\boldsymbol{x}_{3},\dots,\boldsymbol{x}_{d})\in\mathbb{R}^{n_{1}}$ becomes a vector. Similarly, $\mathcal{T}(\bullet,\bullet,\boldsymbol{x}_{3},\boldsymbol{x}_{4},\dots,\boldsymbol{x}_{d})\in\mathbb{R}^{n_{1}\times n_{2}}$ is a matrix, and so on.

Definition 3.2

For a given tensor $\mathcal{T}\in\mathbb{R}^{n_{1}\times n_{2}\times\dots\times n_{d}}$, the spectral norm of $\mathcal{T}$ is defined as

[TABLE]

The tensor spectral norm was proposed by Lim [24] in terms of singular values of a tensor. In light of (3.1), $\|\mathcal{T}\|_{\sigma}$ is the maximal value of the Frobenius inner product between $\mathcal{T}$ and a rank-one tensor whose Frobenius norm is one since $\|\boldsymbol{x}_{1}\otimes\boldsymbol{x}_{2}\otimes\dots\otimes\boldsymbol{x}_{d}\|=\prod_{k=1}^{d}\|\boldsymbol{x}_{k}\|=1$.

When $d=2$, (3.2) is reduced to the matrix spectral norm or the largest singular value of the matrix, which can be computed in polynomial time (e.g., via singular value decompositions). He et al. [13] showed that (3.2) is NP-hard when $d\geq 3$. They also proposed the first polynomial-time algorithm with a worst-case approximation bound $\left(\prod_{k=1}^{d-2}\frac{1}{n_{k}}\right)^{\frac{1}{2}}$. The best known approximation bound for the tensor spectral norm is $\Omega\left(\left(\prod_{k=1}^{d-2}\frac{\ln n_{k}}{n_{k}}\right)^{\frac{1}{2}}\right)$ by So [34]. However, the method in [34] relies on the equivalence between convex optimization and membership oracle queries using the ellipsoid method and it is computationally impractical. There is also a simple but randomized algorithm for the same best bound proposed in [12]. Here in this subsection we are able to present the first easily implementable and deterministic algorithm based on sphere covering, with the same approximation bound $\Omega\left(\left(\prod_{k=1}^{d-2}\frac{\ln n_{k}}{n_{k}}\right)^{\frac{1}{2}}\right)$. To make an exact bound without involving $\Omega$, we need to use $\mathbb{H}^{n}_{5}(2+\sqrt{5},3+\sqrt{5})\in\mathbb{T}\left(n,0.3\sqrt{\frac{\ln n}{n}},O(n^{3})\right)$ in (2.14).

Algorithm 3.3

Given $\mathcal{T}\in\mathbb{R}^{n_{1}\times n_{2}\times\dots\times n_{d}}$, find approximate spectral norm of $\mathcal{T}$.

Enumerate $\boldsymbol{z}_{k}\in\mathbb{H}^{n_{k}}_{5}(2+\sqrt{5},3+\sqrt{5})$ for $k=1,2,\dots,d-2$ and solve the resulting matrix spectral norm problem $\max\left\{\mathcal{T}(\boldsymbol{z}_{1},\boldsymbol{z}_{2},\dots,\boldsymbol{z}_{d-2},\boldsymbol{x}_{d-1},\boldsymbol{x}_{d}):\|\boldsymbol{x}_{d-1}\|=\|\boldsymbol{x}_{d}\|=1\right\}$ whose optimal solution is denoted by $(\boldsymbol{z}_{d-1},\boldsymbol{z}_{d})$. 2. 2.

Compare all the objective values in the first step and output the largest one.

It is obvious that Algorithm 3.3 runs in polynomial time as $|\mathbb{H}^{n_{k}}_{5}(2+\sqrt{5},3+\sqrt{5})|=O({n_{k}}^{3})$ and the matrix spectral norm is polynomial-time computable. Moreover, the corresponding approximate solution $(\boldsymbol{z}_{1},\boldsymbol{z}_{2},\dots,\boldsymbol{z}_{d-2})$ is universal, i.e., $\boldsymbol{z}_{k}\in\mathbb{H}^{n_{k}}_{5}(2+\sqrt{5},3+\sqrt{5})$ is independent of the data $\mathcal{T}$ for $k=1,2,\dots,d-2$.

Theorem 3.4

Algorithm 3.3 is a deterministic polynomial-time algorithm that approximates $\|\mathcal{T}\|_{\sigma}$ with a worst-case approximation bound $0.3^{d-2}\left(\prod_{k=1}^{d-2}\frac{\ln n_{k}}{n_{k}}\right)^{\frac{1}{2}}$ for any $\mathcal{T}\in\mathbb{R}^{n_{1}\times n_{2}\times\dots\times n_{d}}$, i.e., $\boldsymbol{z}_{k}\in\mathbb{H}^{n_{k}}_{5}(2+\sqrt{5},3+\sqrt{5})$ for $k=1,2,\dots,d-2$ and $\boldsymbol{z}_{k}\in\mathbb{S}^{n_{k}}$ for $k=d-1,d$ can be found such that

[TABLE]

Proof. Let us denote $\tau_{k}=0.3\sqrt{\frac{\ln n_{k}}{n_{k}}}$ for $k=1,2,\dots,d-2$. Let $(\boldsymbol{y}_{1},\boldsymbol{y}_{2},\dots,\boldsymbol{y}_{d})$ be an optimal solution of (3.2), i.e., $\mathcal{T}(\boldsymbol{y}_{1},\boldsymbol{y}_{2},\dots,\boldsymbol{y}_{d})=\|\mathcal{T}\|_{\sigma}$. For the vector $\boldsymbol{v}_{1}=\mathcal{T}(\bullet,\boldsymbol{y}_{2},\boldsymbol{y}_{3},\dots,\boldsymbol{y}_{d})$, either $\|\boldsymbol{v}_{1}\|=0$ or there exists $\boldsymbol{z}_{1}\in\mathbb{H}^{n_{1}}_{5}(2+\sqrt{5},3+\sqrt{5})$ such that $\boldsymbol{z}_{1}^{\textnormal{T}}\frac{\boldsymbol{v}_{1}}{\|\boldsymbol{v}_{1}\|}\geq\tau_{1}$. In any case, one has

[TABLE]

Similarly, for every $k=2,3\dots,d-2$ that are chosen one by one increasingly, there exists $\boldsymbol{z}_{k}\in\mathbb{H}^{n_{k}}_{5}(2+\sqrt{5},3+\sqrt{5})$ such that

[TABLE]

where $\boldsymbol{v}_{k}=\mathcal{T}(\boldsymbol{z}_{1},\dots,\boldsymbol{z}_{k-1},\bullet,\boldsymbol{y}_{k+1},\dots,\boldsymbol{y}_{d})$. By applying the above inequalities recursively, we obtain

[TABLE]

The first step of Algorithm 3.3 must have enumerated this $(\boldsymbol{z}_{1},\boldsymbol{z}_{2},\dots,\boldsymbol{z}_{d-2})$ and computed corresponding $\boldsymbol{z}_{d-1}\in\mathbb{S}^{n_{d-1}}$ and $\boldsymbol{z}_{d}\in\mathbb{S}^{n_{d}}$, such that

[TABLE]

Finally, the best one found by the second step must be no less than the above $\mathcal{T}(\boldsymbol{z}_{1},\boldsymbol{z}_{2},\dots,\boldsymbol{z}_{d})$. $\Box$

A closely related problem to the tensor spectral norm (3.2) is sphere constrained homogenous polynomial optimization $\max\left\{p(\boldsymbol{x}):\|\boldsymbol{x}\|=1\right\}$ where $p(\boldsymbol{x})$ a homogenous polynomial function of degree $d$. In other words, there is a symmetric (entries are invariant under permutations of indices) tensor $\mathcal{T}\in\mathbb{R}^{n\times n\times\dots\times n}$ of order $d$ such that $p(\boldsymbol{x})=\mathcal{T}(\boldsymbol{x},\boldsymbol{x},\dots,\boldsymbol{x})$. This is a widely applicable optimization problem but is also NP-hard when the degree of the polynomial $d\geq 3$ [26]. The current best approximation bound for this problem is $\Omega\left(\left(\frac{\ln n}{n}\right)^{d/2-1}\right)$, obtained by a randomized algorithm [12] or a deterministic but not implementable algorithm [34] as it relies on the equivalence between convex optimization and membership oracle queries using the ellipsoid method. In fact, it is not difficult to obtain an easily implementable deterministic algorithm with the same best approximation bound with the help of a polarization formula [13, Lemma 1] below.

Lemma 3.5

Let $\mathcal{T}\in\mathbb{R}^{n\times n\times\dots\times n}$ be a symmetric tensor of order $d$ and $p(\boldsymbol{x})=\mathcal{T}(\boldsymbol{x},\boldsymbol{x},\dots,\boldsymbol{x})$. If $\xi_{i},\xi_{2},\dots,\xi_{d}$ are i.i.d. symmetric Bernoulli random variables (taking values $\pm 1$ with equal probability), then

[TABLE]

We only state the results but leave the details to interested readers.

Theorem 3.6

Let $p(\boldsymbol{x})$ be a homogenous polynomial function of dimension $n$ and degree $d\geq 3$. If $d$ is odd, then there is a deterministic polynomial-time approximation algorithm which outputs $\boldsymbol{z}\in\mathbb{S}^{n}$, such that

[TABLE]

If $d$ is even, then there is a deterministic polynomial-time approximation algorithm which outputs $\boldsymbol{z}\in\mathbb{S}^{n}$, such that

[TABLE]

3.2 Approximation bound for tensor nuclear norm

We now study the approximation for the tensor nuclear norm.

Definition 3.7

For a given tensor $\mathcal{T}\in\mathbb{R}^{n_{1}\times n_{2}\times\dots\times n_{d}}$, the nuclear norm of $\mathcal{T}$ is defined as

[TABLE]

From (3.3), we see that the tensor nuclear norm is the minimum of the sum of Frobenius norms of rank-one tensors in any CP decomposition. A CP decomposition of $\mathcal{T}$ that attains $\|\mathcal{T}\|_{*}$ is called a nuclear decomposition of $\mathcal{T}$ [10]. When $d=2$, the tensor nuclear norm is reduced to the matrix nuclear norm, which is the sum of all singular values. Similar to the role of matrix nuclear norm used in many matrix rank minimization problems, the tensor nuclear norm is the convex envelope of the tensor rank and is widely used in tensor completions [11, 38].

The tensor nuclear norm is the dual norm to the tensor spectral norm, and vice versa, whose proof can be found in [25, 5].

Lemma 3.8

For given tensors $\mathcal{T}$ and $\mathcal{Z}$ in a same tensor space, it follows that

[TABLE]

Computing the tensor nuclear norm is also NP-hard when $d\geq 3$ showed by Friedland and Lim [10]. In fact, it is much harder than computing the tensor spectral norm. From the definition (3.3) finding a CP decomposition is not an easy task for a given $r$, and from the dual formulation (3.4) checking the feasibility $\|\mathcal{Z}\|_{\sigma}\leq 1$ is also NP-hard. Perhaps the only known method is due to Nie [27], which is based on the sum-of-squares relaxation and can only work for symmetric tensors of low dimensions. In terms of polynomial-time approximation bounds, the best bound is $\prod_{k=1}^{d-2}\frac{1}{\sqrt{n_{k}}}$. There are two methods to achieve this bound, one is via matrix flattenings of the tensor [18] and the other is via partitioning the tensor into matrix slices [22]. This bound is worse than the best one for the tensor spectral norm. Let us now bridge the gaps using an idea similar to grid sampling in [17].

To better illustrate our main idea, we discuss the details for a tensor $\mathcal{T}\in\mathbb{R}^{n_{1}\times n_{2}\times n_{3}}$ of order three. According to the dual formulation (3.4),

[TABLE]

Notice that for a given $\boldsymbol{x}$, the constraint $\max_{\|\boldsymbol{y}\|=\|\boldsymbol{z}\|=1}\mathcal{Z}(\boldsymbol{x},\boldsymbol{y},\boldsymbol{z})\leq 1$ is the same to $\|\mathcal{Z}(\boldsymbol{x},\bullet,\bullet)\|_{\sigma}\leq 1$ or the largest singular value of the matrix $\mathcal{Z}(\boldsymbol{x},\bullet,\bullet)$ is no more than one. This can be equivalently represented by $I\succeq\mathcal{Z}(\boldsymbol{x},\bullet,\bullet)\mathcal{Z}(\boldsymbol{x},\bullet,\bullet)^{\textnormal{T}}$. Here a symmetric matrix $A\succeq O$ where $O$ is a zero matrix means that $A$ is positive semidefinite and $A\succeq B$ means that $A-B\succeq O$. Applying the Schur complement, we then have

[TABLE]

By combining with (3.5) we obtain an equivalent formulation of the tensor nuclear norm

[TABLE]

Obviously there is no way to enumerate all $\boldsymbol{x}$ in $\mathbb{S}^{n_{1}}$ in (3.6) but the sphere covering is indeed helpful in this scenario. If we replace $\|\boldsymbol{x}\|=1$ with $\boldsymbol{x}\in\mathbb{H}^{n_{1}}$ for some deterministic $\mathbb{H}^{n_{1}}\in\mathbb{T}(n_{1},\tau,O({n_{1}}^{\alpha}))$ with some university constant $\alpha$, (3.6) is then relaxed to

[TABLE]

This becomes a semidefinte program with $O({n_{1}}^{\alpha})$ number of positive semidefinite constraints.

Algorithm 3.9

Given $\mathcal{T}\in\mathbb{R}^{n_{1}\times n_{2}\times n_{3}}$, find approximate nuclear norm of $\mathcal{T}$.

Choose a $\tau$-hitting set $\mathbb{H}^{n_{1}}\in\mathbb{T}(n_{1},\tau,O({n_{1}}^{\alpha}))$ and solve the semidefinite program

[TABLE] 2. 2.

Output $\tau u$.

Theorem 3.10

For any $\mathbb{H}^{n_{1}}\in\mathbb{T}(n_{1},\tau,O({n_{1}}^{\alpha}))$, Algorithm 3.9 is a deterministic polynomial-time algorithm that approximates $\|\mathcal{T}\|_{*}$ with a worst-case approximation bound $\tau$.

Proof. Denote $\mathcal{Y}$ to be an optimal solution of (3.7). It is easy to see that (3.7) is a relaxation of the maximization problem (3.6) since $\mathbb{H}^{n_{1}}\subseteq\mathbb{S}^{n_{1}}$. Therefore, $u=\langle\mathcal{T},\mathcal{Y}\rangle\geq\|\mathcal{T}\|_{*}$.

For any $\boldsymbol{y},\boldsymbol{z}$, denote $\boldsymbol{v}=\mathcal{Y}(\bullet,\boldsymbol{y},\boldsymbol{z})$ and we have either $\|\boldsymbol{v}\|=0$ or there exists $\boldsymbol{x}\in\mathbb{H}^{n_{1}}$ such that $\boldsymbol{x}^{\textnormal{T}}\frac{\boldsymbol{v}}{\|\boldsymbol{v}\|}\geq\tau$, both leading to $\mathcal{Y}(\boldsymbol{x},\boldsymbol{y},\boldsymbol{z})=\boldsymbol{x}^{\textnormal{T}}\boldsymbol{v}\geq\tau\|\boldsymbol{v}\|=\tau\|\mathcal{Y}(\bullet,\boldsymbol{y},\boldsymbol{z})\|$. Therefore,

[TABLE]

By the feasibility of $\mathcal{Y}$ in (3.7), $\|\mathcal{Y}(\boldsymbol{x},\bullet,\bullet)\|_{\sigma}\leq 1$ for all $\boldsymbol{x}\in\mathbb{H}^{n_{1}}$, implying that

[TABLE]

This means that $\tau\mathcal{Y}$ is a feasible solution to the dual formulation (3.4), and so

[TABLE]

$\Box$

Compared to Algorithm 3.3 that requires (possibly large) enumeration and then comparison, Algorithm 3.9 only needs to solve one semidefinite program, albeit the size is large if $\mathbb{H}^{n_{1}}$ is large. We emphasize that $\mathbb{H}^{n_{1}}$ in Algorithm 3.9 needs to be a deterministic $\tau$-hitting set in order to archive a feasible solution of $\|\mathcal{Z}\|_{\sigma}\leq 1$ in (3.4) with the desired approximation bound $\tau$ in Theorem 3.10. Although a randomized hitting set $\mathbb{H}^{n_{1}}_{1}(\gamma,\epsilon)$ can be used in Algorithm 3.9, it is likely that $\tau\mathcal{Y}$ in the proof of Theorem 3.10 is not feasible to (3.4). However, $\langle\mathcal{T},\mathcal{Y}\rangle$ could still be a good upper bound of $\|\mathcal{T}\|_{*}$ in this case. Let us now extend Algorithm 3.9 to a general tensor of order $d$.

Algorithm 3.11

Given $\mathcal{T}\in\mathbb{R}^{n_{1}\times n_{2}\times\dots\times n_{d}}$, find approximate nuclear norm of $\mathcal{T}$.

Choose $\mathbb{H}^{n_{k}}\in\mathbb{T}(n_{k},\tau_{k},O({n_{k}}^{\alpha_{k}}))$ for $k=1,2,\dots,d-2$ and solve the semidefinite program

[TABLE] 2. 2.

Output $u\prod_{k=1}^{d-2}\tau_{k}$.

We state the final theorem that obtains an improved approximation bound for the tensor nuclear norm using the hitting set $\mathbb{H}^{n}_{5}(2+\sqrt{5},3+\sqrt{5})$ in (2.14). This bound finally matches the current best one for the tensor spectral norm; see Theorem 3.4. The proof is similar to that of Theorem 3.10 and is omitted.

Theorem 3.12

By choosing $\mathbb{H}^{n_{k}}_{5}(2+\sqrt{5},3+\sqrt{5})$ for $k=1,2,\dots,d-2$, Algorithm 3.11 is a deterministic polynomial-time algorithm that approximates $\|\mathcal{T}\|_{*}$ with a worst-case approximation bound $0.3^{d-2}\left(\prod_{k=1}^{d-2}\frac{\ln n_{k}}{n_{k}}\right)^{\frac{1}{2}}$ for any $\mathcal{T}\in\mathbb{R}^{n_{1}\times n_{2}\times\dots\times n_{d}}$.

3.3 Numerical performance of approximation methods

We now test the numerical performance of the methods for approximating tensors norms, in complement to the theoretical results established earlier. All the experiments are conducted on a linux server (Ubuntu 20.04) with an Intel Xeon Platinum 8358 @ 2.60GHz and 512GB of RAM. The computations are implemented in Python 3. The semidefinite optimization solver222https://docs.mosek.com/latest/pythonfusion/tutorial-sdo-shared.html in COPT Fusion API for Python 9.3.13 is called whenever semidefinite programs are involved.

We first test Algorithm 3.3 to approximate the tensor spectral norm using examples in Nie and Wang [28, Examples 3.12, 3.13 and 3.14]. The semidefinite relaxation method in [28] works well in practice and usually finds optimal values. This also enable us to check the true approximation bounds in practice rather than the conservative theoretical bounds. The results for the first two examples are shown in Table 5. For Example 3.13, the method in [28] calls the fmincon function in MATLAB for a local improvement. This is the benchmark optimal value used to compute the approximation bounds. We also apply the classic alternating least square (ALS) method [21] as a local improvement starting from the approximate solutions obtained by Algorithm 3.3. Whenever a local improvement method is applied, the corresponding indicator is appended with a ‘+’ sign.

The results for Example 3.14 in [28] are shown in Table 6. In this example, the method in [28] obtained global optimality directly without applying the local improvement. We also listed the theoretical approximation bound $\sqrt{\frac{\ln n}{n}}$ (without showing the constant disguised under the $\Omega$) of our algorithm for comparison.

Observed from the numerical results of these three examples, Algorithm 3.3 obviously fails to obtain optimality in contrast to a practical method, but with the help of the ALS method the global optimality is obtained for all the test instances. The approximation bounds calculated by these numerical instances are better than the theoretical approximation bounds shown in Section 3.1. In terms of the computational time by comparing with the method in [28], Algorithm 3.3 runs quicker for low dimensions but the time increases quickly when the dimension of the problem increases.

To systematically verify and compare with the theoretical approximation bounds obtained by our algorithms, we now test randomly generated tensors whose spectral and nuclear norms can be easily obtained. In particular, let

[TABLE]

where $(\boldsymbol{x}_{i}^{\textnormal{T}}\boldsymbol{x}_{j})(\boldsymbol{y}_{i}^{\textnormal{T}}\boldsymbol{y}_{j})=\boldsymbol{z}_{i}^{\textnormal{T}}\boldsymbol{z}_{j}=0$ if $i\neq j$. This is a special type of orthogonally decomposable tensors. With the special structure of $\mathcal{T}$ in (3.8), it is not difficulty to see that $\|\mathcal{T}\|_{\sigma}=\max_{1\leq i\leq r}\lambda_{i}$ and $\|\mathcal{T}\|_{*}=\sum_{i=1}^{r}\lambda_{i}$. The components of $\mathcal{T}$ in (3.8), $\lambda_{i}$’s, $\boldsymbol{x}_{i}$’s, $\boldsymbol{y}_{i}$’s and $\boldsymbol{z}_{i}$’s, are generated from i.i.d. standard normal distributions and made positive (by taking the absolute value) or orthogonal if necessary.

We apply Algorithm 3.3 to approximate the spectral norm and Algorithm 3.9 to approximate the nuclear norm for $n\times 10\times 10$ tensors and $10\times n\times n$ tensors, both with varying $n$. Instead of the deterministic hitting set $\mathbb{H}_{5}$ used in the original algorithms, we replace it with a randomized hitting set $\mathbb{H}_{1}$ that is numerically more stable and efficient. The results are shown in Tables 7 and 8 for the spectral norm and in Tables 9 and 10 for the nuclear norm. For each type of tensors with a fixed size, say $5\times 10\times 10$, we randomly generate 200 instances and find an approximate solution of the spectral norm by Algorithm 3.3, whose approximation bound is then computed since the optimal value is known. We then use the approximate solution as a starting point to apply the ALS method as a local improvement. As before, the corresponding indicator is appended with a ‘+’ sign when a local improvement is involved. The same setting is implemented for the tensor nuclear norm by Algorithm 3.9 except that (1) there is no local improvement method to improve our approximation solution and (2) we do not multiply $\tau$ to the output solution $\mathcal{Y}$ as $\tau$ involves an $\Omega$ but directly use $\langle\mathcal{T},\mathcal{Y}\rangle$ to obtain the bound (see the proof of Theorem 3.10) and so the bound is larger than one. In this scenario, the closer to one the better the bound.

From the above tables, we see that the exact approximation bounds obtained by numerical instances outperform the theoretical bound $\Omega\left({\sqrt{\frac{\ln n}{n}}}\right)$ for both the spectral and nuclear norms. For the latter, it obviously beats the previous known best one $\Omega\left(\frac{1}{\sqrt{n}}\right)$. For the spectral norm, running the ALS method starting with our approximate solutions can lead to global optimality for most random generated tensor instances.

4 Concluding remarks

We constructed hitting sets or collections of spherical caps to cover the unit sphere with adjustable parameters for different levels of approximations and cardinalities, listed roughly in Table 1. These readily available products can be used for various decision making problems on spheres or related problems. By applying the covering results we proposed easily implementable and deterministic algorithms to approximate the tensor spectral norm with the current known best approximation bound. The algorithms can be extended to provide approximate solutions for sphere constrained homogeneous polynomial optimization problems. Deterministic algorithms with an improved approximation bound for the tensor nuclear norm were proposed as well. This newly improved bound attains the best known one for the tensor spectral norm.

For $1\leq p\leq\infty$, the tensor spectral $p$-norm [24] generalizes the tensor spectral norm in which the unit sphere $\|\boldsymbol{x}\|=1$ is replaced by the $L_{p}$-sphere $\|\boldsymbol{x}\|_{p}=1$. The tensor nuclear $p$-norm can also be defined similarly [10]. Hou and So [16] studied related $L_{p}$-sphere constrained homogeneous polynomial optimization problems and proposed approximation bounds. It is natural to ask whether one can construct $L_{p}$-sphere coverings and apply them to approximate the tensor spectral and nuclear $p$-norms. The answer is likely true but still challenging. In fact, one can construct randomized hitting sets using similar ideas in Section 2.1 to show the $L_{p}$ version of Theorem 2.2 but deterministic constructions remain difficult. Perhaps a more interesting problem is to explicitly construct hitting sets for the binary hypercube $\{1,-1\}^{n}$ with different levels of approximations and cardinalities. It will have wider applications, particularly in discrete optimization and graph theory. We leave these to future works.

Acknowledgments

The research is partially supported by the National Natural Science Foundation of China (Grants 71771141, 72171141, 71825003, 72150001, 72192832 and 11831002) and Program for Innovative Research Team of Shanghai University of Finance and Economics. The authors would like to thank the anonymous referees for their insightful comments that helped to improve this paper from its original version.