Multi-Label Learning with Global and Local Label Correlation

TL;DR

This paper introduces GLOCAL, a multi-label learning method that simultaneously exploits global and local label correlations, effectively handling both complete and partial label data through latent label representations and manifold optimization.

Contribution

It proposes a novel approach that models both global and local label correlations in multi-label learning, addressing partial label issues and improving correlation estimation.

Findings

Effective on full-label data

Handles missing labels well

Outperforms existing methods

Abstract

It is well-known that exploiting label correlations is important to multi-label learning. Existing approaches either assume that the label correlations are global and shared by all instances; or that the label correlations are local and shared only by a data subset. In fact, in the real-world applications, both cases may occur that some label correlations are globally applicable and some are shared only in a local group of instances. Moreover, it is also a usual case that only partial labels are observed, which makes the exploitation of the label correlations much more difficult. That is, it is hard to estimate the label correlations when many labels are absent. In this paper, we propose a new multi-label approach GLOCAL dealing with both the full-label and the missing-label cases, exploiting global and local label correlations simultaneously, through learning a latent label representation and optimizing label manifolds. The extensive experimental studies validate the effectiveness of our approach on both full-label and missing-label data.

Figures

Tables

Table 1. Table 1: Datasets used in the experiments (“ # # \# instance” is the number of instances, “ # # \# dim” is the feature dimensionality, “ # # \# label” is the total size of the class label set, and “ # # \# label/instance” is the average number of labels possessed by each instance).

	#instance	#dim	#label	#label/instance		#instance	#dim	#label	#label/instance
Arts	5,000	462	26	1.64	Business	5,000	438	30	1.59
Computers	5,000	681	33	1.51	Education	5,000	550	33	1.46
Entertainment	5,000	640	21	1.42	Health	5,000	612	32	1.66
Recreation	5,000	606	22	1.42	Reference	5,000	793	33	1.17
Science	5,000	743	40	1.45	Social	5,000	1,047	39	1.28
Society	5,000	636	27	1.69	Enron	1,702	1,001	53	3.37
Corel5k	5,000	499	374	3.52	Image	2,000	294	5	1.24

Table 2. Table 2: Results for learning with full labels. ↑ ↑ \uparrow ( ↓ ↓ \downarrow ) denotes the larger (smaller) the better. ∙ ∙ \bullet indicates that GLOCAL is significantly better (paired t-tests at 95% significance level).

	Measure	BR	MLLOC	LEML	ML-LRC	GLOCAL
Arts	Rkl ($\downarrow$)	0.201$\pm$0.005$\bullet$	0.177$\pm$0.013$\bullet$	0.170$\pm$0.005$\bullet$	0.157$\pm$0.002$\bullet$	0.138$\pm$0.002
	Auc ($\uparrow$)	0.799$\pm$0.006$\bullet$	0.823$\pm$0.013$\bullet$	0.833$\pm$0.005$\bullet$	0.843$\pm$0.001	0.846$\pm$0.005
	Cvg ($\downarrow$)	7.347$\pm$0.196$\bullet$	6.762$\pm$0.344$\bullet$	6.337$\pm$0.243$\bullet$	5.529$\pm$0.037$\bullet$	5.347$\pm$0.146
	Ap ($\uparrow$)	0.594$\pm$0.006$\bullet$	0.606$\pm$0.006$\bullet$	0.590$\pm$0.005$\bullet$	0.600$\pm$0.007$\bullet$	0.619$\pm$0.005
Business	Rkl ($\downarrow$)	0.072$\pm$0.005$\bullet$	0.055$\pm$0.009$\bullet$	0.056$\pm$0.005$\bullet$	0.044$\pm$0.002	0.044$\pm$0.002
	Auc ($\uparrow$)	0.928$\pm$0.005$\bullet$	0.944$\pm$0.008$\bullet$	0.945$\pm$0.005$\bullet$	0.950$\pm$0.005	0.955$\pm$0.003
	Cvg ($\downarrow$)	4.087$\pm$0.268$\bullet$	3.265$\pm$0.464$\bullet$	3.187$\pm$0.270$\bullet$	2.560$\pm$0.059	2.559$\pm$0.169
	Ap ($\uparrow$)	0.861$\pm$0.007$\bullet$	0.878$\pm$0.011$\bullet$	0.867$\pm$0.007$\bullet$	0.870$\pm$0.005$\bullet$	0.883$\pm$0.004
Computers	Rkl ($\downarrow$)	0.146$\pm$0.007$\bullet$	0.134$\pm$0.014$\bullet$	0.138$\pm$0.004$\bullet$	0.107$\pm$0.002	0.107$\pm$0.002
	Auc ($\uparrow$)	0.854$\pm$0.007$\bullet$	0.866$\pm$0.014$\bullet$	0.895$\pm$0.002	0.894$\pm$0.002	0.895$\pm$0.002
	Cvg ($\downarrow$)	6.654$\pm$0.236$\bullet$	6.224$\pm$0.480$\bullet$	6.148$\pm$0.183$\bullet$	4.893$\pm$0.142	4.889$\pm$0.058
	Ap ($\uparrow$)	0.680$\pm$0.007$\bullet$	0.689$\pm$0.009$\bullet$	0.669$\pm$0.007$\bullet$	0.689$\pm$0.005$\bullet$	0.698$\pm$0.004
Education	Rkl ($\downarrow$)	0.203$\pm$0.010$\bullet$	0.158$\pm$0.021$\bullet$	0.145$\pm$0.008$\bullet$	0.099$\pm$0.002$\bullet$	0.095$\pm$0.002
	Auc ($\uparrow$)	0.797$\pm$0.102$\bullet$	0.842$\pm$0.022$\bullet$	0.859$\pm$0.008$\bullet$	0.868$\pm$0.006	0.878$\pm$0.006
	Cvg ($\downarrow$)	8.979$\pm$0.487$\bullet$	7.381$\pm$0.765$\bullet$	6.711$\pm$0.364$\bullet$	4.531$\pm$0.104	4.529$\pm$0.206
	Ap ($\uparrow$)	0.580$\pm$0.010$\bullet$	0.613$\pm$0.004$\bullet$	0.596$\pm$0.009$\bullet$	0.600$\pm$0.007$\bullet$	0.628$\pm$0.009
Entertainment	Rkl ($\downarrow$)	0.185$\pm$0.006$\bullet$	0.146$\pm$0.013$\bullet$	0.154$\pm$0.005$\bullet$	0.130$\pm$0.005$\bullet$	0.108$\pm$0.004
	Auc ($\uparrow$)	0.815$\pm$0.006$\bullet$	0.854$\pm$0.013$\bullet$	0.852$\pm$0.005$\bullet$	0.871$\pm$0.003	0.874$\pm$0.005
	Cvg ($\downarrow$)	5.006$\pm$0.160$\bullet$	4.293$\pm$0.344$\bullet$	4.193$\pm$0.139$\bullet$	3.505$\pm$0.125$\bullet$	3.114$\pm$0.110
	Ap ($\uparrow$)	0.662$\pm$0.009$\bullet$	0.670$\pm$0.005$\bullet$	0.647$\pm$0.007$\bullet$	0.661$\pm$0.012$\bullet$	0.681$\pm$0.008
Health	Rkl ($\downarrow$)	0.113$\pm$0.001$\bullet$	0.093$\pm$0.005$\bullet$	0.091$\pm$0.003$\bullet$	0.071$\pm$0.003$\bullet$	0.065$\pm$0.002
	Auc ($\uparrow$)	0.886$\pm$0.003$\bullet$	0.907$\pm$0.005$\bullet$	0.913$\pm$0.004$\bullet$	0.929$\pm$0.009	0.923$\pm$0.007
	Cvg ($\downarrow$)	6.193$\pm$0.059$\bullet$	5.403$\pm$0.157$\bullet$	5.063$\pm$0.128$\bullet$	3.751$\pm$0.128	3.858$\pm$0.131
	Ap ($\uparrow$)	0.763$\pm$0.002$\bullet$	0.777$\pm$0.004$\bullet$	0.750$\pm$0.003$\bullet$	0.755$\pm$0.006$\bullet$	0.782$\pm$0.001
Recreation	Rkl ($\downarrow$)	0.197$\pm$0.003$\bullet$	0.184$\pm$0.015$\bullet$	0.185$\pm$0.001$\bullet$	0.170$\pm$0.004$\bullet$	0.155$\pm$0.002
	Auc ($\uparrow$)	0.802$\pm$0.003$\bullet$	0.816$\pm$0.015$\bullet$	0.822$\pm$0.002$\bullet$	0.833$\pm$0.004$\bullet$	0.840$\pm$0.000
	Cvg ($\downarrow$)	5.506$\pm$0.089$\bullet$	5.268$\pm$0.333$\bullet$	5.110$\pm$0.040$\bullet$	4.515$\pm$0.045$\bullet$	4.431$\pm$0.048
	Ap ($\uparrow$)	0.609$\pm$0.005$\bullet$	0.620$\pm$0.004$\bullet$	0.595$\pm$0.004$\bullet$	0.604$\pm$0.003$\bullet$	0.625$\pm$0.004
Reference	Rkl ($\downarrow$)	0.155$\pm$0.005$\bullet$	0.138$\pm$0.008$\bullet$	0.137$\pm$0.004$\bullet$	0.092$\pm$0.003$\bullet$	0.086$\pm$0.003
	Auc ($\uparrow$)	0.845$\pm$0.005$\bullet$	0.862$\pm$0.008$\bullet$	0.872$\pm$0.004$\bullet$	0.900$\pm$0.006	0.894$\pm$0.004
	Cvg ($\downarrow$)	6.171$\pm$0.219$\bullet$	5.514$\pm$0.309$\bullet$	5.277$\pm$0.171$\bullet$	3.438$\pm$0.133	3.387$\pm$0.118
	Ap ($\uparrow$)	0.685$\pm$0.005$\bullet$	0.688$\pm$0.003	0.667$\pm$0.003$\bullet$	0.667$\pm$0.007$\bullet$	0.688$\pm$0.007
Science	Rkl ($\downarrow$)	0.197$\pm$0.009$\bullet$	0.166$\pm$0.017$\bullet$	0.170$\pm$0.005$\bullet$	0.131$\pm$0.002$\bullet$	0.118$\pm$0.003
	Auc ($\uparrow$)	0.802$\pm$0.010$\bullet$	0.834$\pm$0.018	0.834$\pm$0.005$\bullet$	0.860$\pm$0.003	0.853$\pm$0.010
	Cvg ($\downarrow$)	10.189$\pm$0.435$\bullet$	8.867$\pm$0.751$\bullet$	8.885$\pm$0.197$\bullet$	6.704$\pm$0.122$\bullet$	6.434$\pm$0.137
	Ap ($\uparrow$)	0.568$\pm$0.012$\bullet$	0.581$\pm$0.009	0.551$\pm$0.008$\bullet$	0.561$\pm$0.009$\bullet$	0.580$\pm$0.009
Social	Rkl ($\downarrow$)	0.112$\pm$0.001$\bullet$	0.094$\pm$0.013$\bullet$	0.106$\pm$0.006$\bullet$	0.075$\pm$0.005	0.075$\pm$0.005
	Auc ($\uparrow$)	0.888$\pm$0.002$\bullet$	0.906$\pm$0.013$\bullet$	0.894$\pm$0.006$\bullet$	0.917$\pm$0.005	0.915$\pm$0.005
	Cvg ($\downarrow$)	6.036$\pm$0.125$\bullet$	5.147$\pm$0.401$\bullet$	5.521$\pm$0.301$\bullet$	4.651$\pm$0.102	4.537$\pm$0.258
	Ap ($\uparrow$)	0.724$\pm$0.005$\bullet$	0.764$\pm$0.008	0.731$\pm$0.005$\bullet$	0.719$\pm$0.003$\bullet$	0.758$\pm$0.008
Society	Rkl ($\downarrow$)	0.204$\pm$0.004$\bullet$	0.182$\pm$0.006$\bullet$	0.182$\pm$0.007$\bullet$	0.142$\pm$0.002$\bullet$	0.136$\pm$0.005
	Auc ($\uparrow$)	0.796$\pm$0.005$\bullet$	0.818$\pm$0.006$\bullet$	0.822$\pm$0.008$\bullet$	0.840$\pm$0.006	0.844$\pm$0.006
	Cvg ($\downarrow$)	8.048$\pm$0.108$\bullet$	7.392$\pm$0.216$\bullet$	7.438$\pm$0.162$\bullet$	5.973$\pm$0.108	5.852$\pm$0.194
	Ap ($\uparrow$)	0.610$\pm$0.007$\bullet$	0.623$\pm$0.004$\bullet$	0.599$\pm$0.006$\bullet$	0.605$\pm$0.006$\bullet$	0.633$\pm$0.009
Enron	Rkl ($\downarrow$)	0.194$\pm$0.006$\bullet$	0.169$\pm$0.012$\bullet$	0.159$\pm$0.005$\bullet$	0.133$\pm$0.004$\bullet$	0.125$\pm$0.004
	Auc ($\uparrow$)	0.806$\pm$0.006$\bullet$	0.831$\pm$0.009$\bullet$	0.851$\pm$0.006$\bullet$	0.869$\pm$0.004$\bullet$	0.877$\pm$0.005
	Cvg ($\downarrow$)	23.618$\pm$0.450$\bullet$	21.724$\pm$0.950$\bullet$	18.531$\pm$0.707$\bullet$	16.654$\pm$0.198	16.737$\pm$0.622
	Ap ($\uparrow$)	0.575$\pm$0.006$\bullet$	0.586$\pm$0.009$\bullet$	0.600$\pm$0.004$\bullet$	0.591$\pm$0.004$\bullet$	0.647$\pm$0.006
Corel5k	Rkl ($\downarrow$)	0.271$\pm$0.006$\bullet$	0.230$\pm$0.012$\bullet$	0.246$\pm$0.004$\bullet$	0.170$\pm$0.002	0.173$\pm$0.005
	Auc ($\uparrow$)	0.699$\pm$0.006$\bullet$	0.757$\pm$0.012$\bullet$	0.754$\pm$0.005$\bullet$	0.825$\pm$0.005	0.827$\pm$0.005
	Cvg ($\downarrow$)	261.99$\pm$3.15$\bullet$	201.80$\pm$6.71$\bullet$	184.58$\pm$1.72$\bullet$	137.31$\pm$2.49	136.91$\pm$3.21
	Ap ($\uparrow$)	0.153$\pm$0.001$\bullet$	0.182$\pm$0.005$\bullet$	0.188$\pm$0.004$\bullet$	0.198$\pm$0.003	0.200$\pm$0.004
Image	Rkl ($\downarrow$)	0.181$\pm$0.011	0.180$\pm$0.008	0.181$\pm$0.012	0.180$\pm$0.009	0.179$\pm$0.004
	Auc ($\uparrow$)	0.812$\pm$0.011	0.810$\pm$0.012	0.786$\pm$0.005$\bullet$	0.748$\pm$0.010$\bullet$	0.819$\pm$0.009
	Cvg ($\downarrow$)	1.004$\pm$0.050	0.975$\pm$0.060	1.000$\pm$0.027	1.000$\pm$0.019	0.975$\pm$0.054
	Ap ($\uparrow$)	0.788$\pm$0.008	0.794$\pm$0.010	0.790$\pm$0.008	0.790$\pm$0.010	0.795$\pm$0.007

Table 3. Table 3: Results for learning with full labels on the small clusters (each containing fewer than 5 % percent 5 5\% of the samples). ↑ ↑ \uparrow ( ↓ ↓ \downarrow ) denotes the larger (smaller) the better. ∙ ∙ \bullet indicates that GLOCAL is significantly better (paired t-tests at 95% significance level).

		GLObal	loCAL	GLOCAL			GLObal	loCAL	GLOCAL
Art	Rkl ($\downarrow$)	0.137$\pm$0.003$\bullet$	0.137$\pm$0.002$\bullet$	0.130$\pm$0.005	Bus	Rkl ($\downarrow$)	0.040$\pm$0.002	0.040$\pm$0.002	0.040$\pm$0.003
	Auc ($\uparrow$)	0.863$\pm$0.003$\bullet$	0.863$\pm$0.002$\bullet$	0.870$\pm$0.005		Auc ($\uparrow$)	0.958$\pm$0.003	0.958$\pm$0.003	0.958$\pm$0.003
	Cvg ($\downarrow$)	5.286$\pm$0.046$\bullet$	5.286$\pm$0.046$\bullet$	5.197$\pm$0.065		Cvg ($\downarrow$)	2.529$\pm$0.035	2.528$\pm$0.040	2.528$\pm$0.040
	Ap ($\uparrow$)	0.602$\pm$0.013$\bullet$	0.602$\pm$0.010$\bullet$	0.631$\pm$0.011		Ap ($\uparrow$)	0.882$\pm$0.002$\bullet$	0.882$\pm$0.002$\bullet$	0.886$\pm$0.003
Com	Rkl ($\downarrow$)	0.095$\pm$0.002$\bullet$	0.095$\pm$0.002$\bullet$	0.092$\pm$0.002	Edu	Rkl ($\downarrow$)	0.101$\pm$0.002$\bullet$	0.101$\pm$0.002$\bullet$	0.097$\pm$0.002
	Auc ($\uparrow$)	0.905$\pm$0.002$\bullet$	0.905$\pm$0.002$\bullet$	0.908$\pm$0.001		Auc ($\uparrow$)	0.899$\pm$0.002$\bullet$	0.899$\pm$0.002$\bullet$	0.903$\pm$0.002
	Cvg ($\downarrow$)	4.482$\pm$0.032$\bullet$	4.486$\pm$0.040$\bullet$	4.364$\pm$0.055		Cvg ($\downarrow$)	4.803$\pm$0.033$\bullet$	4.805$\pm$0.036$\bullet$	4.672$\pm$0.051
	Ap ($\uparrow$)	0.677$\pm$0.003	0.676$\pm$0.003	0.678$\pm$0.005		Ap ($\uparrow$)	0.605$\pm$0.003$\bullet$	0.605$\pm$0.003$\bullet$	0.624$\pm$0.005
Ent	Rkl ($\downarrow$)	0.091$\pm$0.002$\bullet$	0.091$\pm$0.002$\bullet$	0.086$\pm$0.003	Hea	Rkl ($\downarrow$)	0.054$\pm$0.002	0.054$\pm$0.003	0.053$\pm$0.004
	Auc ($\uparrow$)	0.909$\pm$0.002$\bullet$	0.909$\pm$0.002$\bullet$	0.914$\pm$0.002		Auc ($\uparrow$)	0.945$\pm$0.003	0.946$\pm$0.003	0.947$\pm$0.003
	Cvg ($\downarrow$)	2.817$\pm$0.027$\bullet$	2.797$\pm$0.035$\bullet$	2.709$\pm$0.059		Cvg ($\downarrow$)	3.508$\pm$0.036	3.506$\pm$0.049	3.504$\pm$0.041
	Ap ($\uparrow$)	0.748$\pm$0.003$\bullet$	0.749$\pm$0.004$\bullet$	0.759$\pm$0.006		Ap ($\uparrow$)	0.810$\pm$0.004	0.810$\pm$0.004	0.812$\pm$0.006
Rec	Rkl ($\downarrow$)	0.124$\pm$0.002$\bullet$	0.124$\pm$0.002$\bullet$	0.118$\pm$0.002	Ref	Rkl ($\downarrow$)	0.060$\pm$0.002$\bullet$	0.061$\pm$0.003$\bullet$	0.054$\pm$0.004
	Auc ($\uparrow$)	0.871$\pm$0.003	0.870$\pm$0.003	0.872$\pm$0.004		Auc ($\uparrow$)	0.940$\pm$0.003$\bullet$	0.939$\pm$0.004$\bullet$	0.946$\pm$0.004
	Cvg ($\downarrow$)	3.704$\pm$0.033	3.700$\pm$0.037	3.700$\pm$0.042		Cvg ($\downarrow$)	2.552$\pm$0.043$\bullet$	2.559$\pm$0.057$\bullet$	2.325$\pm$0.060
	Ap ($\uparrow$)	0.670$\pm$0.004	0.670$\pm$0.004	0.672$\pm$0.005		Ap ($\uparrow$)	0.739$\pm$0.004$\bullet$	0.739$\pm$0.004$\bullet$	0.783$\pm$0.005
Sci	Rkl ($\downarrow$)	0.107$\pm$0.004	0.108$\pm$0.004	0.107$\pm$0.004	Soc	Rkl ($\downarrow$)	0.063$\pm$0.002$\bullet$	0.063$\pm$0.002$\bullet$	0.060$\pm$0.002
	Auc ($\uparrow$)	0.893$\pm$0.004	0.892$\pm$0.004	0.893$\pm$0.005		Auc ($\uparrow$)	0.930$\pm$0.002$\bullet$	0.930$\pm$0.002$\bullet$	0.934$\pm$0.002
	Cvg ($\downarrow$)	5.937$\pm$0.041$\bullet$	5.941$\pm$0.049$\bullet$	5.845$\pm$0.054		Cvg ($\downarrow$)	3.558$\pm$0.033	3.559$\pm$0.038	3.552$\pm$0.049
	Ap ($\uparrow$)	0.608$\pm$0.003	0.608$\pm$0.003	0.610$\pm$0.003		Ap ($\uparrow$)	0.797$\pm$0.002	0.797$\pm$0.003	0.798$\pm$0.003
Soci	Rkl ($\downarrow$)	0.126$\pm$0.003$\bullet$	0.126$\pm$0.005$\bullet$	0.113$\pm$0.005	Enr	Rkl ($\downarrow$)	0.117$\pm$0.002$\bullet$	0.119$\pm$0.003$\bullet$	0.105$\pm$0.005
	Auc ($\uparrow$)	0.874$\pm$0.003$\bullet$	0.874$\pm$0.004$\bullet$	0.887$\pm$0.005		Auc ($\uparrow$)	0.883$\pm$0.004$\bullet$	0.881$\pm$0.004$\bullet$	0.895$\pm$0.004
	Cvg ($\downarrow$)	5.554$\pm$0.047$\bullet$	5.553$\pm$0.053$\bullet$	5.208$\pm$0.059		Cvg ($\downarrow$)	19.440$\pm$0.833$\bullet$	19.372$\pm$0.915$\bullet$	17.511$\pm$1.231
	Ap ($\uparrow$)	0.670$\pm$0.004$\bullet$	0.670$\pm$0.005$\bullet$	0.711$\pm$0.005		Ap ($\uparrow$)	0.685$\pm$0.005$\bullet$	0.673$\pm$0.005$\bullet$	0.706$\pm$0.007
Cor	Rkl ($\downarrow$)	0.163$\pm$0.002$\bullet$	0.163$\pm$0.002$\bullet$	0.160$\pm$0.002	Ima	Rkl ($\downarrow$)	0.197$\pm$0.003$\bullet$	0.199$\pm$0.004$\bullet$	0.190$\pm$0.004
	Auc ($\uparrow$)	0.837$\pm$0.002$\bullet$	0.837$\pm$0.002$\bullet$	0.840$\pm$0.002		Auc ($\uparrow$)	0.803$\pm$0.003$\bullet$	0.801$\pm$0.003$\bullet$	0.810$\pm$0.003
	Cvg ($\downarrow$)	130.84$\pm$1.01$\bullet$	131.13$\pm$1.21$\bullet$	128.40$\pm$1.30		Cvg ($\downarrow$)	1.064$\pm$0.015$\bullet$	1.066$\pm$0.021$\bullet$	1.027$\pm$0.027
	Ap ($\uparrow$)	0.212$\pm$0.003	0.212$\pm$0.003	0.214$\pm$0.005		Ap ($\uparrow$)	0.764$\pm$0.003$\bullet$	0.763$\pm$0.004$\bullet$	0.771$\pm$0.005

Table 4. Table 4: Recovery results for missing label data on ranking loss(Rkl), average auc(Auc), coverage(Cvg) and average precision(Ap).. ↑ ↑ \uparrow ( ↓ ↓ \downarrow ) denotes the larger (smaller) the better. ∙ ∙ \bullet indicates that the GLOCAL is significantly better (paired t-tests at 95% significance level).

	Measure	$\rho$	MAXIDE	LEML	ML-LRC	GLOCAL		Measure	$\rho$	MAXIDE	LEML	ML-LRC	GLOCAL
Art	Rkl ($\downarrow$)	30	0.131$\bullet$	0.133$\bullet$	0.137$\bullet$	0.103	Bus	Rkl ($\downarrow$)	30	0.044$\bullet$	0.046$\bullet$	0.046$\bullet$	0.029
		70	0.083$\bullet$	0.090$\bullet$	0.083$\bullet$	0.074			70	0.026$\bullet$	0.027$\bullet$	0.024$\bullet$	0.021
	Auc ($\uparrow$)	30	0.871$\bullet$	0.848$\bullet$	0.879$\bullet$	0.897		Auc ($\uparrow$)	30	0.956$\bullet$	0.954$\bullet$	0.954$\bullet$	0.971
		70	0.918$\bullet$	0.912$\bullet$	0.910$\bullet$	0.928			70	0.974$\bullet$	0.973$\bullet$	0.974$\bullet$	0.979
	Cvg ($\downarrow$)	30	5.195$\bullet$	5.231$\bullet$	5.161$\bullet$	4.189		Cvg ($\downarrow$)	30	2.550$\bullet$	2.622$\bullet$	2.622$\bullet$	1.830
		70	3.616$\bullet$	3.733$\bullet$	3.778$\bullet$	3.234			70	1.742$\bullet$	1.783$\bullet$	1.746$\bullet$	1.477
	Ap ($\uparrow$)	30	0.645$\bullet$	0.634$\bullet$	0.640$\bullet$	0.652		Ap ($\uparrow$)	30	0.876$\bullet$	0.878$\bullet$	0.876$\bullet$	0.893
		70	0.720	0.720	0.709$\bullet$	0.720			70	0.905$\bullet$	0.901$\bullet$	0.903$\bullet$	0.908
Com	Rkl ($\downarrow$)	30	0.101$\bullet$	0.098$\bullet$	0.097$\bullet$	0.073	Edu	Rkl ($\downarrow$)	30	0.097$\bullet$	0.093$\bullet$	0.089$\bullet$	0.069
		70	0.059$\bullet$	0.063$\bullet$	0.061$\bullet$	0.052			70	0.061$\bullet$	0.061$\bullet$	0.061$\bullet$	0.058
	Auc ($\uparrow$)	30	0.905$\bullet$	0.908$\bullet$	0.909$\bullet$	0.933		Auc ($\uparrow$)	30	0.902$\bullet$	0.907$\bullet$	0.911$\bullet$	0.932
		70	0.947$\bullet$	0.943$\bullet$	0.945$\bullet$	0.955			70	0.938$\bullet$	0.938$\bullet$	0.940	0.942
	Cvg ($\downarrow$)	30	4.627$\bullet$	4.586$\bullet$	4.565$\bullet$	3.511		Cvg ($\downarrow$)	30	4.672$\bullet$	4.372$\bullet$	3.914$\bullet$	3.171
		70	2.912$\bullet$	3.100$\bullet$	3.095$\bullet$	2.586			70	3.113$\bullet$	3.106$\bullet$	3.000	2.815
	Ap ($\uparrow$)	30	0.709$\bullet$	0.700$\bullet$	0.705$\bullet$	0.726		Ap ($\uparrow$)	30	0.653	0.648$\bullet$	0.653	0.655
		70	0.787	0.787	0.787	0.787			70	0.711	0.702$\bullet$	0.710	0.711
Ent	Rkl ($\downarrow$)	30	0.104$\bullet$	0.103$\bullet$	0.106$\bullet$	0.085	Hea	Rkl ($\downarrow$)	30	0.060$\bullet$	0.057$\bullet$	0.054$\bullet$	0.041
		70	0.063	0.063	0.063	0.062			70	0.037$\bullet$	0.036$\bullet$	0.032	0.030
	Auc ($\uparrow$)	30	0.898$\bullet$	0.899$\bullet$	0.899$\bullet$	0.916		Auc ($\uparrow$)	30	0.941$\bullet$	0.943$\bullet$	0.947$\bullet$	0.960
		70	0.940	0.938	0.940	0.940			70	0.964$\bullet$	0.964$\bullet$	0.968	0.971
	Cvg ($\downarrow$)	30	3.058$\bullet$	2.994$\bullet$	3.022$\bullet$	2.512		Cvg ($\downarrow$)	30	3.577$\bullet$	3.462$\bullet$	3.465$\bullet$	2.567
		70	1.987	2.051	2.080	1.957			70	2.524$\bullet$	2.465$\bullet$	2.450$\bullet$	2.152
	Ap ($\uparrow$)	30	0.704	0.698$\bullet$	0.698$\bullet$	0.704		Ap ($\uparrow$)	30	0.796$\bullet$	0.794$\bullet$	0.798	0.801
		70	0.763$\bullet$	0.765	0.765	0.768			70	0.848	0.842$\bullet$	0.848	0.848
Rec	Rkl ($\downarrow$)	30	0.130$\bullet$	0.133$\bullet$	0.135$\bullet$	0.110	Ref	Rkl ($\downarrow$)	30	0.083$\bullet$	0.083$\bullet$	0.083$\bullet$	0.063
		70	0.078$\bullet$	0.080$\bullet$	0.080$\bullet$	0.068			70	0.048	0.049	0.049	0.048
	Auc ($\uparrow$)	30	0.873$\bullet$	0.870$\bullet$	0.869$\bullet$	0.895		Auc ($\uparrow$)	30	0.919$\bullet$	0.919$\bullet$	0.918$\bullet$	0.939
		70	0.925$\bullet$	0.923$\bullet$	0.920$\bullet$	0.934			70	0.955	0.953	0.953	0.955
	Cvg ($\downarrow$)	30	3.899$\bullet$	3.919$\bullet$	4.048$\bullet$	3.291		Cvg ($\downarrow$)	30	3.436$\bullet$	3.392$\bullet$	3.372$\bullet$	2.520
		70	2.560$\bullet$	2.607$\bullet$	2.620$\bullet$	2.262			70	2.039$\bullet$	2.103$\bullet$	2.195$\bullet$	1.972
	Ap ($\uparrow$)	30	0.680$\bullet$	0.663$\bullet$	0.660$\bullet$	0.681		Ap ($\uparrow$)	30	0.681	0.664$\bullet$	0.674	0.679
		70	0.767$\bullet$	0.763$\bullet$	0.760$\bullet$	0.770			70	0.745	0.746	0.746	0.746
Sci	Rkl ($\downarrow$)	30	0.110$\bullet$	0.111$\bullet$	0.110$\bullet$	0.086	Soc	Rkl ($\downarrow$)	30	0.069$\bullet$	0.069$\bullet$	0.063$\bullet$	0.042
		70	0.063	0.071$\bullet$	0.070$\bullet$	0.063			70	0.041$\bullet$	0.040$\bullet$	0.040$\bullet$	0.026
	Auc ($\uparrow$)	30	0.889$\bullet$	0.889$\bullet$	0.889$\bullet$	0.913		Auc ($\uparrow$)	30	0.930$\bullet$	0.930$\bullet$	0.936$\bullet$	0.957
		70	0.935	0.928$\bullet$	0.923$\bullet$	0.935			70	0.964$\bullet$	0.959$\bullet$	0.966$\bullet$	0.973
	Cvg ($\downarrow$)	30	6.193$\bullet$	6.141$\bullet$	6.271$\bullet$	4.845		Cvg ($\downarrow$)	30	3.865$\bullet$	3.920$\bullet$	3.304$\bullet$	2.443
		70	3.771	3.914$\bullet$	3.878$\bullet$	3.751			70	2.103$\bullet$	2.386$\bullet$	2.373$\bullet$	1.663
	Ap ($\uparrow$)	30	0.615	0.613	0.614	0.615		Ap ($\uparrow$)	30	0.780$\bullet$	0.780$\bullet$	0.784$\bullet$	0.802
		70	0.689$\bullet$	0.647$\bullet$	0.650$\bullet$	0.691			70	0.854$\bullet$	0.865	0.865	0.865
Soci	Rkl ($\downarrow$)	30	0.129$\bullet$	0.128$\bullet$	0.123$\bullet$	0.102	Enr	Rkl ($\downarrow$)	30	0.091$\bullet$	0.115$\bullet$	0.085$\bullet$	0.075
		70	0.074	0.081$\bullet$	0.073	0.073			70	0.042	0.060$\bullet$	0.040	0.040
	Auc ($\uparrow$)	30	0.871$\bullet$	0.872$\bullet$	0.877$\bullet$	0.898		Auc ($\uparrow$)	30	0.910$\bullet$	0.887$\bullet$	0.918$\bullet$	0.926
		70	0.926	0.919$\bullet$	0.928	0.929			70	0.960	0.942$\bullet$	0.962	0.962
	Cvg ($\downarrow$)	30	5.557$\bullet$	5.459$\bullet$	5.167$\bullet$	4.496		Cvg ($\downarrow$)	30	14.24$\bullet$	16.65$\bullet$	13.45$\bullet$	12.05
		70	3.641$\bullet$	3.824$\bullet$	3.608$\bullet$	3.442			70	7.961$\bullet$	10.33$\bullet$	7.480	7.510
	Ap ($\uparrow$)	30	0.646	0.629$\bullet$	0.650	0.652		Ap ($\uparrow$)	30	0.739	0.711$\bullet$	0.739	0.739
		70	0.719	0.717	0.719	0.719			70	0.854	0.842$\bullet$	0.855	0.855
Cor	Rkl ($\downarrow$)	30	0.226$\bullet$	0.214$\bullet$	0.206$\bullet$	0.185	Ima	Rkl ($\downarrow$)	30	0.302$\bullet$	0.184$\bullet$	0.175	0.173
		70	0.138$\bullet$	0.131$\bullet$	0.123	0.125			70	0.251$\bullet$	0.148	0.148	0.148
	Auc ($\uparrow$)	30	0.773$\bullet$	0.786$\bullet$	0.794$\bullet$	0.814		Auc ($\uparrow$)	30	0.820$\bullet$	0.828	0.826	0.828
		70	0.874	0.874	0.874	0.874			70	0.834$\bullet$	0.857	0.855	0.855
	Cvg ($\downarrow$)	30	204.90$\bullet$	182.76$\bullet$	178.60$\bullet$	153.82		Cvg ($\downarrow$)	30	1.493$\bullet$	1.104$\bullet$	0.967	0.950
		70	103.63	102.42	102.30	102.30			70	0.790$\bullet$	0.760	0.770	0.760
	Ap ($\uparrow$)	30	0.275	0.259$\bullet$	0.275	0.275		Ap ($\uparrow$)	30	0.739$\bullet$	0.776$\bullet$	0.775$\bullet$	0.785
		70	0.279	0.279	0.279	0.279			70	0.768$\bullet$	0.841	0.834	0.841

Table 5. Table 5: Prediction results for missing label data on ranking loss(Rkl), average auc(Auc), coverage(Cvg) and average precision(Ap).. ↑ ↑ \uparrow ( ↓ ↓ \downarrow ) denotes the larger (smaller) the better. ∙ ∙ \bullet indicates that the GLOCAL is significantly better (paired t-tests at 95% significance level).

	Measure	$\rho$	MMLLOC	LEML	ML-LRC	GLOCAL		Measure	$\rho$	MMLLOC	LEML	ML-LRC	GLOCAL
Art	Rkl ($\downarrow$)	30	0.225$\bullet$	0.204$\bullet$	0.184$\bullet$	0.144	Bus	Rkl ($\downarrow$)	30	0.083$\bullet$	0.063$\bullet$	0.061$\bullet$	0.054
		70	0.193$\bullet$	0.181$\bullet$	0.159$\bullet$	0.139			70	0.064$\bullet$	0.058$\bullet$	0.046	0.046
	Auc ($\uparrow$)	30	0.781$\bullet$	0.801$\bullet$	0.828	0.831		Auc ($\uparrow$)	30	0.917$\bullet$	0.928$\bullet$	0.937	0.937
		70	0.819$\bullet$	0.825$\bullet$	0.838	0.840			70	0.935$\bullet$	0.942$\bullet$	0.950	0.952
	Cvg ($\downarrow$)	30	9.033$\bullet$	7.369$\bullet$	6.281$\bullet$	5.867		Cvg ($\downarrow$)	30	4.643$\bullet$	3.954$\bullet$	3.279$\bullet$	2.863
		70	7.262$\bullet$	6.431$\bullet$	5.432	5.352			70	3.670$\bullet$	3.303$\bullet$	2.580	2.579
	Ap ($\uparrow$)	30	0.529$\bullet$	0.503$\bullet$	0.517$\bullet$	0.572		Ap ($\uparrow$)	30	0.843$\bullet$	0.866$\bullet$	0.858$\bullet$	0.879
		70	0.583$\bullet$	0.589$\bullet$	0.588$\bullet$	0.607			70	0.861$\bullet$	0.870$\bullet$	0.870$\bullet$	0.881
Com	Rkl ($\downarrow$)	30	0.201$\bullet$	0.179$\bullet$	0.152	0.154	Edu	Rkl ($\downarrow$)	30	0.187$\bullet$	0.176$\bullet$	0.144$\bullet$	0.137
		70	0.150$\bullet$	0.141$\bullet$	0.115	0.113			70	0.165$\bullet$	0.151$\bullet$	0.113	0.111
	Auc ($\uparrow$)	30	0.849$\bullet$	0.880	0.873$\bullet$	0.883		Auc ($\uparrow$)	30	0.815$\bullet$	0.817$\bullet$	0.845	0.846
		70	0.868$\bullet$	0.894	0.895	0.896			70	0.844$\bullet$	0.842$\bullet$	0.860	0.860
	Cvg ($\downarrow$)	30	8.808$\bullet$	7.392$\bullet$	6.052$\bullet$	5.798		Cvg ($\downarrow$)	30	11.089$\bullet$	9.672$\bullet$	6.350	6.338
		70	6.871$\bullet$	6.306$\bullet$	5.000	4.976			70	8.096$\bullet$	7.595$\bullet$	5.075	5.070
	Ap ($\uparrow$)	30	0.631$\bullet$	0.646$\bullet$	0.636$\bullet$	0.669		Ap ($\uparrow$)	30	0.538$\bullet$	0.537$\bullet$	0.543$\bullet$	0.592
		70	0.674$\bullet$	0.665$\bullet$	0.667$\bullet$	0.691			70	0.586$\bullet$	0.591$\bullet$	0.600$\bullet$	0.622
Ent	Rkl ($\downarrow$)	30	0.229$\bullet$	0.175$\bullet$	0.152$\bullet$	0.122	Hea	Rkl ($\downarrow$)	30	0.137$\bullet$	0.095$\bullet$	0.085	0.085
		70	0.164$\bullet$	0.159$\bullet$	0.129$\bullet$	0.109			70	0.109$\bullet$	0.074$\bullet$	0.071$\bullet$	0.065
	Auc ($\uparrow$)	30	0.832$\bullet$	0.826$\bullet$	0.849$\bullet$	0.859		Auc ($\uparrow$)	30	0.894$\bullet$	0.896$\bullet$	0.907	0.906
		70	0.842	0.850$\bullet$	0.870	0.871			70	0.901$\bullet$	0.920	0.920	0.920
	Cvg ($\downarrow$)	30	6.029$\bullet$	5.755$\bullet$	4.170	4.153		Cvg ($\downarrow$)	30	7.104$\bullet$	6.248$\bullet$	4.924	4.814
		70	4.857$\bullet$	4.643$\bullet$	3.483$\bullet$	3.117			70	5.866$\bullet$	5.167$\bullet$	3.960	3.963
	Ap ($\uparrow$)	30	0.601$\bullet$	0.601$\bullet$	0.601$\bullet$	0.645		Ap ($\uparrow$)	30	0.727$\bullet$	0.715$\bullet$	0.720$\bullet$	0.752
		70	0.635$\bullet$	0.645$\bullet$	0.643$\bullet$	0.670			70	0.762$\bullet$	0.770$\bullet$	0.766$\bullet$	0.775
Rec	Rkl ($\downarrow$)	30	0.266$\bullet$	0.245$\bullet$	0.202$\bullet$	0.165	Ref	Rkl ($\downarrow$)	30	0.199$\bullet$	0.187$\bullet$	0.137$\bullet$	0.098
		70	0.204$\bullet$	0.196$\bullet$	0.167$\bullet$	0.156			70	0.155$\bullet$	0.145$\bullet$	0.098$\bullet$	0.086
	Auc ($\uparrow$)	30	0.785$\bullet$	0.828$\bullet$	0.802$\bullet$	0.839		Auc ($\uparrow$)	30	0.851$\bullet$	0.847$\bullet$	0.868$\bullet$	0.886
		70	0.800$\bullet$	0.837$\bullet$	0.836$\bullet$	0.845			70	0.861$\bullet$	0.869$\bullet$	0.895	0.898
	Cvg ($\downarrow$)	30	7.084$\bullet$	6.842$\bullet$	5.397$\bullet$	4.545		Cvg ($\downarrow$)	30	7.549$\bullet$	6.463$\bullet$	5.052$\bullet$	3.367
		70	5.952$\bullet$	5.685$\bullet$	4.490	4.430			70	6.419$\bullet$	6.130$\bullet$	3.694$\bullet$	3.348
	Ap ($\uparrow$)	30	0.547$\bullet$	0.540$\bullet$	0.540$\bullet$	0.573		Ap ($\uparrow$)	30	0.631	0.609$\bullet$	0.611$\bullet$	0.638
		70	0.597$\bullet$	0.567$\bullet$	0.600$\bullet$	0.614			70	0.675	0.653$\bullet$	0.653$\bullet$	0.672
Sci	Rkl ($\downarrow$)	30	0.257$\bullet$	0.203$\bullet$	0.169$\bullet$	0.144	Soc	Rkl ($\downarrow$)	30	0.149$\bullet$	0.089$\bullet$	0.095$\bullet$	0.075
		70	0.189$\bullet$	0.174$\bullet$	0.134	0.129			70	0.108$\bullet$	0.079$\bullet$	0.076$\bullet$	0.073
	Auc ($\uparrow$)	30	0.827$\bullet$	0.827$\bullet$	0.830$\bullet$	0.837		Auc ($\uparrow$)	30	0.906$\bullet$	0.906$\bullet$	0.905$\bullet$	0.913
		70	0.840$\bullet$	0.849	0.850	0.850			70	0.910$\bullet$	0.900$\bullet$	0.914	0.914
	Cvg ($\downarrow$)	30	12.805$\bullet$	10.587$\bullet$	8.794$\bullet$	6.809		Cvg ($\downarrow$)	30	7.652$\bullet$	7.567$\bullet$	6.308	6.088
		70	9.960$\bullet$	9.501$\bullet$	6.900$\bullet$	6.416			70	5.886$\bullet$	5.386$\bullet$	5.103	4.929
	Ap ($\uparrow$)	30	0.503$\bullet$	0.479$\bullet$	0.485$\bullet$	0.531		Ap ($\uparrow$)	30	0.712$\bullet$	0.682$\bullet$	0.700$\bullet$	0.738
		70	0.569	0.551$\bullet$	0.570$\bullet$	0.574			70	0.748$\bullet$	0.719$\bullet$	0.728$\bullet$	0.761
Soci	Rkl ($\downarrow$)	30	0.252$\bullet$	0.202$\bullet$	0.175$\bullet$	0.139	Enr	Rkl ($\downarrow$)	30	0.179$\bullet$	0.172$\bullet$	0.173$\bullet$	0.149
		70	0.208$\bullet$	0.194$\bullet$	0.141$\bullet$	0.136			70	0.170$\bullet$	0.162$\bullet$	0.152$\bullet$	0.129
	Auc ($\uparrow$)	30	0.804$\bullet$	0.808$\bullet$	0.826	0.826		Auc ($\uparrow$)	30	0.820$\bullet$	0.830$\bullet$	9,843$\bullet$	0.853
		70	0.816$\bullet$	0.816$\bullet$	0.840	0.840			70	0.829$\bullet$	0.839$\bullet$	0.849$\bullet$	0.872
	Cvg ($\downarrow$)	30	9.550$\bullet$	8.637$\bullet$	6.944$\bullet$	5.816		Cvg ($\downarrow$)	30	22.72$\bullet$	21.41$\bullet$	20.42$\bullet$	19.01
		70	8.227$\bullet$	7.638$\bullet$	5.750	5.750			70	21.90$\bullet$	19.53$\bullet$	18.17$\bullet$	17.16
	Ap ($\uparrow$)	30	0.569$\bullet$	0.563$\bullet$	0.565$\bullet$	0.601		Ap ($\uparrow$)	30	0.580$\bullet$	0.582$\bullet$	0.580$\bullet$	0.589
		70	0.606$\bullet$	0.589$\bullet$	0.590$\bullet$	0.625			70	0.585$\bullet$	0.601$\bullet$	0.607$\bullet$	0.635
Cor	Rkl ($\downarrow$)	30	0.332$\bullet$	0.308$\bullet$	0.331$\bullet$	0.285	Ima	Rkl ($\downarrow$)	30	0.224$\bullet$	0.204$\bullet$	0.220$\bullet$	0.200
		70	0.248$\bullet$	0.250$\bullet$	0.199	0.194			70	0.195$\bullet$	0.188	0.197$\bullet$	0.187
	Auc ($\uparrow$)	30	0.673$\bullet$	0.693$\bullet$	0.670$\bullet$	0.714		Auc ($\uparrow$)	30	0.796$\bullet$	0.795$\bullet$	0.800	0.801
		70	0.747$\bullet$	0.749$\bullet$	0.801	0.805			70	0.812	0.811	0.810	0.813
	Cvg ($\downarrow$)	30	275.41$\bullet$	233.83$\bullet$	240.17$\bullet$	211.84		Cvg ($\downarrow$)	30	1.160$\bullet$	1.103$\bullet$	1.131$\bullet$	1.070
		70	212.84$\bullet$	190.83$\bullet$	160.59$\bullet$	151.23			70	1.066$\bullet$	1.030	1.040$\bullet$	1.025
	Ap ($\uparrow$)	30	0.158$\bullet$	0.166$\bullet$	0.165$\bullet$	0.174		Ap ($\uparrow$)	30	0.745$\bullet$	0.752$\bullet$	0.744$\bullet$	0.760
		70	0.176$\bullet$	0.185$\bullet$	0.188	0.192			70	0.768$\bullet$	0.772	0.770$\bullet$	0.777

Table 6. Table 6: CPU timing results for learning with missing labels ( ρ = 70 𝜌 70 \rho=70 ). F is the time to fill in the missing labels. C is the time for clustering, I is the time for initialization, and R is the time of the main learning procedure. A is the total time (sum of F, I, C and R). Note that some algorithms may not need F, C or I.

	MBR			MMLLOC					LEML			ML-LRC			GLOCAL
	A	F	R	A	F	C	I	R	A	I	R	A	I	R	A	C	I	R
Arts	109	8	101	107	8	1	0	98	34	0	34	87	0	87	47	1	20	26
Business	38	6	32	104	6	1	0	97	35	0	35	82	0	82	49	1	24	24
Computers	78	11	67	121	11	1	0	109	46	0	46	94	0	94	53	1	31	21
Education	60	8	52	115	8	1	0	106	45	0	45	64	0	64	45	1	29	15
Entertainment	66	6	60	91	6	1	0	84	42	0	42	73	0	73	53	2	22	29
Health	64	11	53	116	11	1	0	104	41	0	41	75	0	75	67	1	32	34
Recreation	63	4	59	97	5	1	0	91	46	0	46	55	0	55	51	2	22	27
Reference	75	14	61	131	15	9	0	107	38	0	38	91	0	91	78	8	32	38
Science	101	15	86	133	15	1	0	117	53	0	53	103	0	103	77	2	32	43
Social	163	36	127	149	33	8	0	108	37	0	37	147	0	147	90	7	35	48
Society	83	8	75	106	8	1	0	97	32	0	32	117	0	117	44	2	18	24
Enron	47	10	37	59	10	1	0	48	38	0	38	78	0	78	69	1	25	43
Corel5k	458	272	186	1529	268	1	0	1260	307	0	307	709	0	709	413	1	78	344
Image	5	1	4	25	2	1	0	22	28	0	28	14	0	14	15	1	5	9

Equations

$$\underset{\mathbf{U},\mathbf{V},\mathbf{W}}{\min }\Vert {\Pi }_{\Omega }(\mathbf{Y}-\mathbf{UV}){\Vert }_F^2+\lambda \Vert {\mathbf{V}-\mathbf{W}}^{\top }\mathbf{X}{\Vert }_F^2+{\lambda }_2\mathcal{R}(\mathbf{U},\mathbf{V},\mathbf{W}),$$

$$\underset{\mathbf{U},\mathbf{V},\mathbf{W}}{\min }\Vert {\Pi }_{\Omega }(\mathbf{Y}-\mathbf{UV}){\Vert }_F^2+\lambda \Vert {\mathbf{V}-\mathbf{W}}^{\top }\mathbf{X}{\Vert }_F^2+{\lambda }_2\mathcal{R}(\mathbf{U},\mathbf{V},\mathbf{W})+{\lambda }_3\mathrm{tr}({\mathbf{F}}_0^{\top }{\mathbf{L}}_0{\mathbf{F}}_0)+\sum _{m=1}^g{\lambda }_4\mathrm{tr}({\mathbf{F}}_m^{\top }{\mathbf{L}}_m{\mathbf{F}}_m),$$

$$\underset{\mathbf{U},\mathbf{V},\mathbf{W}}{\min }$$

$$\underset{\mathbf{U},\mathbf{V},\mathbf{W},\mathcal{Z}}{\min }$$

$$\underset{{\mathbf{Z}}_m}{\min }$$

$${\nabla }_{{\mathbf{Z}}_m}=\frac{{\lambda }_3{n}_m}{n}U{\mathbf{W}}^{\top }\mathbf{X}{\mathbf{X}}^{\top }\mathbf{W}{\mathbf{U}}^{\top }{\mathbf{Z}}_m+{\lambda }_4{\mathbf{UW}}^{\top }{\mathbf{X}}_m{\mathbf{X}}_m^{\top }{\mathbf{WU}}^{\top }{\mathbf{Z}}_m.$$

$${\mathbf{z}}_{m,j}\leftarrow {\mathbf{z}}_{m,j}/\Vert {\mathbf{z}}_{m,j}\Vert ,$$

$$\underset{\mathbf{V}}{\min }\Vert \mathbf{J}\circ (\mathbf{Y}-\mathbf{UV}){\Vert }_F^2+\lambda \Vert \mathbf{V}-{\mathbf{W}}^{\top }\mathbf{X}{\Vert }_F^2+{\lambda }_2\Vert \mathbf{V}{\Vert }_F^2.$$

$$\underset{{\mathbf{v}}_i}{\min }\Vert \mathrm{Diag}({\mathbf{j}}_i){\mathbf{y}}_i-\mathrm{Diag}({\mathbf{j}}_i)\mathbf{U}{\mathbf{v}}_i{\Vert }^2+\lambda \Vert {\mathbf{v}}_i-{\mathbf{W}}^{\top }{\mathbf{x}}_i{\Vert }^2+{\lambda }_2\Vert {\mathbf{v}}_i{\Vert }^2.$$

$$\bm{v}_{i}=\big{(}\bm{U}^{\top}\mathrm{Diag}(\bm{j}_{i})\bm{U}+(\lambda+\lambda_{2})\bm{\mathrm{I}}\big{)}^{-1}\big{(}\lambda\bm{W}^{\top}\bm{x}_{i}+\bm{U}^{\top}\mathrm{Diag}(\bm{j}_{i})\bm{y}_{i}\big{)}.$$

$${\nabla }_{\mathbf{V}}={\mathbf{U}}^{\top }\left(\mathbf{J}\circ \left(\mathbf{UV}-\mathbf{Y}\right)\right)+\lambda (\mathbf{V}-{\mathbf{W}}^{\top }\mathbf{X})+{\lambda }_2\mathbf{V}.$$

$$\min\limits_{\bm{U}}\|\bm{J}\circ(\bm{Y}\!-\!\bm{UV})\|_{F}^{2}+\lambda_{2}\|\bm{U}\|_{F}^{2}\!+\!\sum_{m=1}^{g}\!\Big{(}\!\frac{\lambda_{3}n_{m}}{n}\mathrm{tr}(\bm{F}_{0}^{\top}\!\bm{Z}_{m}\!\bm{Z}_{m}^{\top}\!\bm{F}_{0})\!+\!\lambda_{4}\mathrm{tr}(\bm{F}_{m}^{\top}\!\bm{Z}_{m}\!\bm{Z}_{m}^{\top}\!\bm{F}_{m})\!\Big{)}\!.$$

$$\nabla_{\bm{U}}\!=\!(\bm{J}\circ(\bm{UV}-\bm{Y}))\bm{V}^{\top}+\lambda_{2}\bm{U}\!+\sum_{m=1}^{g}\!\!\bm{Z}_{i}\bm{Z}_{i}^{\top}\bm{U}\Big{(}\frac{\lambda_{3}n_{m}}{n}\bm{W}^{\top}\bm{X}_{m}\bm{X}_{m}^{\top}\bm{W}\!+\!\lambda_{4}\bm{W}^{\top}\bm{XX}^{\top}\bm{W}\Big{)}.$$

$$\underset{\mathbf{W}}{\min }\lambda \Vert \mathbf{V}-{\mathbf{W}}^{\top }\mathbf{X}{\Vert }_F^2+{\lambda }_2\Vert \mathbf{W}{\Vert }_F^2+\sum _{m=1}^g\left(\frac{{\lambda }_3{n}_m}{n}\mathrm{tr}({\mathbf{F}}_0^{\top }{\mathbf{Z}}_m{\mathbf{Z}}_m^{\top }{\mathbf{F}}_0)+{\lambda }_4\mathrm{tr}({\mathbf{F}}_m^{\top }{\mathbf{Z}}_m{\mathbf{Z}}_m^{\top }{\mathbf{F}}_m)\right).$$

$$\nabla_{\bm{W}}=\lambda\bm{X}\left(\bm{X}^{\top}\bm{W}-\bm{V}^{\top}\right)+\lambda_{2}\bm{W}+\sum_{m=1}^{g}\Big{(}\frac{\lambda_{3}n_{m}}{n}\bm{XX}^{\top}+\lambda_{4}\bm{X}_{m}\bm{X}_{m}^{\top}\Big{)}\bm{WU}^{\top}\bm{Z}_{m}\bm{Z}_{m}^{\top}\bm{U}.$$

$$\text{Rkl}=\frac{1}{p}\sum _{i=1}^p\frac{|{\mathbf{Q}}_i|}{|{\mathbf{C}}_i^{+}||{\mathbf{C}}_i^{-}|}.$$

$$\text{Auc}=\frac{1}{l}\sum _{j=1}^l\frac{|{\tilde{\mathbf{Q}}}_j|}{|{\mathbf{Z}}_j^{+}||{\mathbf{Z}}_j^{-}|}.$$

$$\text{Cvg}=\frac{1}{p}\sum _{i=1}^p\max \{{\mathrm{rank}}_{\mathbf{f}}({\mathbf{x}}_i,j)|j\in {\mathbf{C}}_i^{+}\}-1.$$

$$\text{Ap}=\frac{1}{p}\sum _{i=1}^p\frac{1}{|{\mathbf{C}}_i^{+}|}{\sum }_{c\in {\mathbf{C}}_i^{+}}\frac{|{\hat{\mathbf{Q}}}_{i,c}|}{{\mathrm{rank}}_{\mathbf{f}}({\mathbf{x}}_i,c)}.$$

Full text

\cortext

[cor1]Corresponding author.

1 Introduction

In real-world classification applications, an instance is often associated with more than one class labels. For example, a scene image can be annotated with several tags boutell2004learning , a document may belong to multiple topics ueda2002parametric , and a piece of music may be associated with different genres turnbull2008semantic . Thus, multi-label learning has attracted a lot of attention in recent years zhang2014review .

Current studies on multi-label learning try to incorporate label correlations of different orders zhang2014review . However, existing approaches mostly focus on global label correlations shared by all instances furnkranz2008multilabel ; ji2008extracting ; read2011classifier . For example, labels “fish” and “ocean” are highly correlated, and so are “stock” and “finance”. On the other hand, certain label correlations are only shared by a local data subset huang2012 . For example, “apple” is related to “fruit” in gourmet magazines, but is related to “digital devices” in technology magazines. Previous studies focus on exploiting either global or local label correlations. However, considering both of them is obviously more beneficial and desirable.

Another problem with label correlations is that they are usually difficult to specify manually. As label correlations may vary in different contexts and there is no unified measure for specifying appropriate correlations, they are usually estimated from the observed data. Some approaches learn the label hierarchies by hierarchical clustering Punera2005Automatically or Bayesian network structure learning zhang2010multi . However, the hierarchical structure may not exist in some applications. For example, labels such as “desert”, “mountains”, “sea”, “sunset” and “trees” do not have any natural hierarchical correlations, and label hierarchies may not be useful. Others estimate label correlations by the co-occurrence of labels in training data NIPS2011_4239 . However, it may cause overfitting. Moreover, co-occurrence is less meaningful for labels with very few positive instances.

In multi-label learning, some labels may be missing from the training set. For example, human labelers may ignore object classes they do not know or of little interest. Recently, multi-label learning with missing labels has become a hot topic. Xu et al. xu2013speedup and Yu et al. Yu2014 considered using the low-rank structure on the instance-label mapping. A more direct approach to model the label dependency approximates the label matrix as a product of two low-rank matrices goldberg2010transduction . This leads to simpler recovery of the missing labels, and produces a latent representation of the label matrix.

In the missing label cases, estimation of label correlation becomes even more difficult, as the observed label distribution is different from the true one. As a result, the aforementioned methods (based on hierarchical clustering and co-occurrence, for example) will produce biased estimates of label correlations.

In this paper, we propose a new approach called “Multi-Label Learning with GLObal and loCAL Correlation” (GLOCAL), which simultaneously recovers the missing labels, trains the linear classifiers and exploits both global and local label correlations. It learns a latent label representation. Classifier outputs are encouraged to be similar on highly positively correlated labels, and dissimilar on highly negatively correlated labels. We do not assume the presence of external knowledge sources specifying the label correlations. Instead, these correlations are learned simultaneously with the latent label representations and instance-label mapping.

The rest of the paper is organized as follows. In Section 2, related works of multi-label learning with label correlations are introduced. In Section 3, the problem formulation and the GLOCAL approach are proposed. Experimental results are presented in Section 4. Finally, Section 5 concludes the work.

Notations For a matrix $\bm{A}$, $\bm{A}^{\top}$ denotes its transpose, $\mathrm{tr(\bm{A})}$ is its trace, $\|\bm{A}\|_{F}$ is its Frobenius norm, and $\text{diag}(\bm{A})$ returns a vector containing the diagonal elements of $\bm{A}$. For two matrices $\bm{A}$ and $\bm{B}$, $\bm{A}\circ\bm{B}$ denotes the Hadamard (element-wise) product. For a vector $\bm{c}$, $\|\bm{c}\|_{2}$ is its $\ell_{2}$-norm, and $\text{Diag}(\bm{c})$ returns a diagonal matrix with $\bm{c}$ on the diagonal.

2 Related Work

Multi-label learning has been widely studied in recent years. Based on the degree of label correlations used, it can be divided into three categories zhang2014review : (i) first-order; (ii) second-order; and (iii) high-order. For the first-order strategy, label correlations are not considered, and the multi-label problem is transformed into multiple independent binary classification problems. For example, BR boutell2004learning trains a classifier for each label independently. For the second-order strategy, pairwise label relations are considered. For example, CLR furnkranz2008multilabel transforms the multi-label learning problem into the pairwise label ranking problem. For the high-order strategy, all other labels’ influences imposed on each label are taken into account. For example, CC read2011classifier transforms the multi-label learning problem into a chain of binary classification problems, with the ground-truth labels encoded into the features.

Most previous studies focus on global label correlations. However, MLLOC huang2012 demonstrates that sometimes label correlations may only be shared by a local data subset. Specifically, it enhances the feature representation of each instance by embedding a code into the feature space, which encodes the influence of labels of an instance to the local label correlations. This has some limitations. First, when the dimensionality of the feature space is large, the code is less discriminative and will be dominated by the original features. Second, MLLOC considers only the local label correlations, but not the global ones. Third, MLLOC cannot learn with missing labels.

In some real-world applications, labels are partially observed, and multi-label learning with missing labels has attracted much attention. MAXIDE xu2013speedup is based on fast low-rank matrix completion, and has strong theoretical guarantees. However, it only works in the transductive setting. Moreover, a label correlation matrix has to be specified manually. LEML Yu2014 also relies on a low-rank structure, and works in an inductive setting. However, it only implicitly uses global label correlations. ML-LRC xu2014learning adopts a low-rank structure to capture global label correlations, and addresses the missing labels by introducing a supplementary label matrix. However, only global label correlations are taken into account. Obviously, it would be more desirable to learn both global and local label correlations simultaneously.

Manifold regularization belkin2006manifold exploits instance similarity by forcing the predicted values on similar instances to be similar. A similar idea can be adapted to the label manifold, and so predicted values for correlated labels should be similar. However, the Laplacian matrix is based on some label similarity or correlation matrix, which can be hard to specify as discussed in Section 1.

3 The Proposed Approach

In multi-label learning, an instance can be associated with multiple class labels. Let $\bm{C}=\{c_{1},\dots,c_{l}\}$ be the class label set of $l$ labels. We denote the feature vector of an instance by $\bm{x}\in\mathcal{X}\subseteq\mathbb{R}^{d}$, and denote the ground-truth label vector by $\tilde{\bm{y}}\in\mathcal{Y}\subseteq\{-1,1\}^{l}$, where $[\tilde{\bm{y}}]_{j}=1$ if $\bm{x}$ is with class label $c_{j}$, and $-1$ otherwise. As mentioned in Section 1, instances in the training data may be partially labeled, i.e., some labels may be missing. We adopt the general setting that both positive and negative labels can be missing goldberg2010transduction ; xu2013speedup ; Yu2014 . The observed label vector is denoted $\bm{y}$, where $[\bm{y}]_{j}=0$ if class label $c_{j}$ is not labeled (i.e. it is missing), and $[\bm{y}]_{j}=[\tilde{\bm{y}}]_{j}$ otherwise. Given the training data $\mathcal{D}=\{(\bm{x}_{i},\bm{y}_{i})\}_{i=1}^{n}$, our goal is to learn a mapping function $\Psi:\mathcal{X}\rightarrow\mathcal{Y}$.

In this paper, we propose the GLOCAL algorithm, which learns and exploits both global and local label correlations via label manifolds. To recover the missing labels, learning of the latent label representation and classifier training are performed simultaneously.

3.1 Basic Model

Let $\tilde{\bm{Y}}\!=\![\tilde{\bm{y}}_{1},\dots,\tilde{\bm{y}}_{n}]\!\in\!\{-1,1\}^{l\times n}$ be the ground-truth label matrix, where each $\tilde{\bm{y}}_{i}$ is the label vector for instance $i$. As discussed in Section 1, $\tilde{\bm{Y}}$ is low-rank. Let its rank be $k<l$. Thus, $\tilde{\bm{Y}}$ can be written as the low-rank decomposition $\bm{UV}$, where $\bm{U}\in\mathbb{R}^{l\times k}$ and $\bm{V}\in\mathbb{R}^{k\times n}$. Intuitively, $\bm{V}$ represents the latent labels that are more compact and more semantically abstract than the original labels, while matrix $\bm{U}$ projects the original labels to the latent label space.

In general, the labels are only partially observed. Let the observed label matrix be $\bm{Y}\!=\![\bm{y}_{1},\dots,\bm{y}_{n}]\!\in\!\{-1,0,1\}^{l\times n}$, and $\Omega$ be the set containing indices of the observed labels in $\bm{Y}$ (i.e., indices of the nonzero elements in $\bm{Y}$). We focus on minimizing the reconstruction error on the observed labels, i.e., $\|\Pi_{\Omega}(\bm{Y-UV})\|_{F}^{2}$, where $[\Pi_{\Omega}(\bm{A})]_{ij}=A_{ij}$ if $\left(i,j\right)\in\Omega$, and 0 otherwise. Moreover, we use a linear mapping $\bm{W}\in\mathbb{R}^{d\times k}$ to map instances to the latent labels. This $\bm{W}$ is learned by minimizing $\|\bm{V-W}^{\top}\bm{X}\|_{F}^{2}$, where $\bm{X}=[\bm{x}_{1},\dots,\bm{x}_{n}]\in\mathbb{R}^{d\times n}$ is the instance matrix. Combining these two, we obtain the following optimization problem:

[TABLE]

where $\mathcal{R}(\bm{U,V,W})$ is a regularizer and $\lambda$, $\lambda_{2}$ are tradeoff parameters. While the square loss has been used in Eqn (1), it can be replaced by any differentiable loss function. The prediction on $\bm{\bm{x}}$ is $\mathrm{sign}(\bm{f}(\bm{x}))$, where $\bm{f}(\bm{x})=\bm{UW}^{\top}\bm{x}$. Let $\bm{f}=[f_{1},\cdots,f_{l}]^{\top}$, thus $f_{j}(\bm{x})$ denotes the predictive value on $j$-th label for $\bm{x}$. We concatenate all $\bm{f}(\bm{x}),\forall\bm{x}\in\bm{X}$, denoted by $F_{0}$, thus $F_{0}=[\bm{f}(\bm{x}_{1}),\cdots,\bm{f}(\bm{x}_{n})]=\bm{UW}^{\top}\bm{X}$.

3.2 Global and Local Manifold Regularizers

Exploiting label correlations is an essential ingredient in multi-label learning. Here, we use label correlations to regularize the model. Intuitively, the more positively correlated two labels are, the closer are the corresponding classifier outputs, and vice versa. Let $\bm{S}_{0}=[S_{ij}]\in\mathbb{R}^{l\times l}$ be the global label correlation matrix. The manifold regularizer $\sum_{i,j}S_{ij}\|\bm{f}_{i,:}-\bm{f}_{j,:}\|_{2}^{2}$ should have a small value melacci2011primallapsvm . Here, $\bm{f}_{i,:}$, the $i$th row of $\bm{F}_{0}$, is the vector of classifier outputs for the $i$th label on the $n$ samples. Let $\bm{D}_{0}$ be the diagonal matrix with diagonal $\bm{S}_{0}\mathbf{1}$, where $\mathbf{1}$ is the vector of ones. The manifold regularizer can be equivalently written as $\mathrm{tr}(\bm{F}_{0}^{\top}\bm{L}_{0}\bm{F}_{0})$ luo2009non , where $\bm{L}_{0}=\bm{D}_{0}-\bm{S}_{0}$ is the Laplacian matrix of $\bm{S}_{0}$.

As discussed in Section 1, label correlations may vary from one local region to another. Assume that the data $\bm{X}$ is partitioned into $g$ groups $\{\bm{X}_{1},\dots,\bm{X}_{g}\}$, where $\bm{X}_{m}\in\mathbb{R}^{d\times n_{m}}$ has size $n_{m}$. This partitioning can be obtained by domain knowledge (e.g., gene pathways subramanian2005gene and networks chuang2007network in bioinformatics applications) or clustering. Let $\bm{Y}_{m}$ be the label submatrix in $\bm{Y}$ corresponding to $\bm{X}_{m}$, and $\bm{S}_{m}\in\mathbb{R}^{l\times l}$ be the local label correlation matrix of group $m$. Similar to global label correlation, to encourage the classifier outputs to be similar on the positively correlated labels and dissimilar on the negatively correlated ones, we minimize $\mathrm{tr}(\bm{F}_{m}^{\top}\bm{L}_{m}\bm{F}_{m})$, where $\bm{L}_{m}$ is the Laplacian matrix of $\bm{S}_{m}$ and $\bm{F}_{m}=\bm{UW}^{\top}\bm{X}_{m}$ is the classifier output matrix for group $m$.

Combining global and local label correlations with Eqn. (1), we have the following optimization problem:

[TABLE]

where $\lambda,\lambda_{2},\lambda_{3},\lambda_{4}$ are tradeoff parameters.

Intuitively, a large local group contributes more to the global label correlations. In particular, the following Lemma shows that when the cosine similarity is used to compute $\bm{S}_{ij}$, we have $\bm{S}_{0}=\sum_{m=1}^{g}\frac{n_{m}}{n}\bm{S}_{m}$.

Lemma 1

Let $[\bm{S}_{0}]_{ij}=\frac{\bm{y}_{i,:}\bm{y}_{j,:}^{\top}}{\|\bm{y}_{i,:}\|\|\bm{y}_{j,:}\|}$ and $[\bm{S}_{m}]_{ij}=\frac{\bm{y}_{m,i,:}\bm{y}_{m,j,:}^{\top}}{\|\bm{y}_{m,i,:}\|\|\bm{y}_{m,j,:}\|}$, where $\bm{y}_{i,:}$ is the $i$th row of $\bm{Y}$, and $\bm{y}_{m,i,:}$ is the $i$th row of $\bm{Y}_{m}$. Then, $\bm{S}_{0}=\sum_{m=1}^{g}\frac{n_{m}}{n}\bm{S}_{m}$.

In general, when the global label correlation matrix is a linear combination of the local label correlation matrices, the following Proposition shows that the global label Laplacian matrix is also a linear combination of the local label Laplacian matrices with the same combination coefficients.

Proposition 1

If $\bm{S}_{0}\!=\!\sum_{m=1}^{g}\beta_{m}\bm{S}_{m}$, then $\bm{L}_{0}\!=\!\sum_{m=1}^{g}\beta_{m}\bm{L}_{m}$.

Using Lemma 1 and Proposition 1, Eqn. (2) can then be rewritten as follows:

[TABLE]

The success of label manifold regularization hinges on a good correlation matrix (or equivalently, a good Laplacian matrix). In multi-label learning, one rudimentary approach is to compute the correlation coefficient between two labels by cosine distance wang2009image . However, this can be noisy since some labels may only have very few positive instances in the training data. When labels can be missing, this computation may even become misleading, since the label distribution of observed labels may be much different from that of the ground-truth label distribution due to the missing labels.

In this paper, instead of specifying any correlation metric or label correlation matrix, we learn the Laplacian matrices directly. Note that the Laplacian matrices are symmetric positive definite. Thus, for $m\in\{1,\ldots,g\}$, we decompose $\bm{L}_{m}$ as $\bm{Z}_{m}\bm{Z}_{m}^{\top}$, where $\bm{Z}_{m}\!\in\!\mathbb{R}^{l\times k}$. For simplicity, $k$ is set to the dimensionality of the latent representation $\bm{V}$. As a result, learning the Laplacian matrices is transformed to learning $\bm{\mathcal{Z}}\equiv\{\bm{Z}_{1},\dots,\bm{Z}_{g}\}$. Note that optimization w.r.t. $\bm{Z}_{m}$ may lead to the trivial solution $\bm{Z}_{m}=\mathbf{0}$. To avoid this problem, we add the constraint that the diagonal entries in $\bm{Z}_{m}\bm{Z}_{m}^{\top}$ are 1, for $m\in\{1,\cdots,g\}$. This constraint also enables us to obtain a normalized Laplacian matrix chung1997spectral of $L_{m}$.

Let $\bm{J}=[\bm{J}_{ij}]$ be the indicator matrix with $\bm{J}_{ij}=1$ if $(i,j)\in\Omega$, and 0 otherwise. $\Pi_{\Omega}(\bm{Y-UV})$ can be rewritten as the Hadamard product $\bm{J}\circ\bm{(Y-UV})$. Combining the decomposition of Laplacian matrices and the diagonal constraints of $\bm{Z}_{m}$, we obtain the optimization problem as:

[TABLE]

Moreover, we will use $\mathcal{R}(\bm{U,V,W})=\|\bm{U}\|_{F}^{2}+\|\bm{V}\|_{F}^{2}+\|\bm{W}\|_{F}^{2}$.

3.3 Learning by Alternating Minimization

Problem (4) can be solved by alternating minimization (Algorithm 1). In each iteration, we update one of the variables in $\{\bm{Z},\bm{U},\bm{V},\bm{W}\}$ with gradient descent, and leave the others fixed. Specifically, the MANOPT toolbox manopt is utilized to implement gradient descent with line search on the Euclidean space for the update of $\bm{U},\bm{V},\bm{W}$, and on the manifolds for the update of $\bm{Z}$.

3.3.1 Updating $\bm{Z}_{m}$

With $\bm{U},\bm{V},\bm{W}$ fixed, problem (4) reduces to

[TABLE]

for each $m\in\{1,\dots,g\}$. Due to the constraint $\mathrm{diag}(\bm{Z}_{m}\bm{Z}_{m}^{\top})=\mathbf{1}$, it has no closed-form solution, and we will solve it with projected gradient descent. The gradient of the objective w.r.t. $\bm{Z}_{m}$ is

[TABLE]

To satisfy the constraint $\mathrm{diag}(\bm{Z}_{m}\bm{Z}_{m}^{\top})=\mathbf{1}$, we project each row of $\bm{Z}_{m}$ onto the unit norm ball after each update:

[TABLE]

where $\bm{z}_{m,j}$ is the $j$th row of $\bm{Z}_{m}$.

3.3.2 Updating $\bm{V}$

With $\bm{Z}_{m}$’s and $\bm{U},\bm{W}$ fixed, problem (4) reduces to

[TABLE]

Notice that each column of $\bm{V}$ is independent to each other, and thus $\bm{V}$ can be solved column-by-column. Let $\bm{j}_{i}$ and $\bm{v}_{i}$ be $i$th column of $\bm{J}$ and $\bm{V}$, respectively. The optimization problem for $\bm{v}_{i}$ can be written as:

[TABLE]

Setting the gradient w.r.t. $\bm{v}_{i}$ to 0, we obtain the following closed-form solution of $\bm{v}_{i}$:

[TABLE]

This involves computing a matrix inverse for each $i$. If this is expensive, we can use gradient descent instead. The gradient of the objective in (6) w.r.t. $\bm{V}$ is

[TABLE]

3.3.3 Updating $\bm{U}$

With $\bm{Z}_{m}$’s and $\bm{V},\bm{W}$ fixed, problem (4) reduces to

[TABLE]

Again, we use gradient descent, and the gradient w.r.t. $\bm{U}$ is:

[TABLE]

3.3.4 Updating $\bm{W}$

With $\bm{Z}_{m}$’s and $\bm{U},\bm{V}$ fixed, problem (4) reduces to

[TABLE]

The gradient w.r.t. $W$ is:

[TABLE]

4 Experiments

In this section, extensive experiments are performed on text and image datasets. Performance on both the full-label and missing-label cases are discussed.

4.1 Setup

4.1.1 Data sets

On text, eleven Yahoo datasets111http://www.kecl.ntt.co.jp/as/members/ueda/yahoo.tar (Arts, Business, Computers, Education, Entertainment, Health, Recreation, Reference, Science, Social and Society) and the Enron dataset222http://mulan.sourceforge.net/datasets-mlc.html are used. On images, the Corel5k33footnotemark: 3 and Image333http://cse.seu.edu.cn/people/zhangml/files/Image.rar datasets are used. In the sequel, each dataset is denoted by its first three letters.444“Society” is denoted “Soci”, so as to distinguish it from “Social”. Detailed information of the datasets are shown in Table 1. For each dataset, we randomly select $60\%$ of the instances for training, and the rest for testing.

4.1.2 Baselines

In the GLOCAL algorithm, we use the kmeans clustering algorithm to partition the data into local groups. The solution of Eqn. (1) is used to warm-start $\bm{U},\bm{V}$ and $\bm{W}$. The $\bm{\mathcal{Z}}_{m}$’s are randomly initialized. GLOCAL is compared with the following state-of-the-art multi-label learning algorithms:

BR boutell2004learning , which trains a binary linear SVM (using the LIBLINEAR package REF08a ) for each label independently; 2. 2.

MLLOC huang2012 , which exploits local label correlations by encoding them into the instance’s feature representation; 3. 3.

LEML Yu2014 , which learns a linear instance-to-label mapping with low-rank structure, and implicitly takes advantage of global label correlation; 4. 4.

ML-LRC xu2014learning , which learns and exploits low-rank global label correlations for multi-label classification with missing labels.

Note that BR does not take label correlation into account. MLLOC considers only local label correlations; LEML implicitly uses global label correlations, whereas ML-LRC models global label correlation directly. On the ability to handle missing labels, BR and MLLOC can only learn with full labels.

For simplicity, we set $\lambda=1$ in GLOCAL. The other parameters, as well as those of the baseline methods, are selected via 5-fold cross-validation on the training set. All the algorithms are implemented in Matlab (with some C++ code for LEML).

4.1.3 Performance Evaluation

Let $p$ be the number of test instances, $\bm{C}_{i}^{+},\bm{C}_{i}^{-}$ be the sets of positive and negative labels associated with the $i$th instance; and $\bm{Z}_{j}^{+},\bm{Z}_{j}^{-}$ be the sets of positive and negative instances belonging to the $j$th label. Given input $\bm{x}$, let $\mathrm{rank}_{\bm{f}}(\bm{x},y)$ be the rank of label $y$ in the predicted label ranking (sorted in descending order). For performance evaluation, we use the following popular metrics in multi-label learning zhang2014review :

Ranking loss (Rkl): This is the fraction that a negative label is ranked higher than a positive label. For instance $i$, define $\bm{Q}_{i}=\{(j^{\prime},j^{\prime\prime})\;|\;f_{j^{\prime}}(\bm{x}_{i})\leq f_{j^{\prime\prime}}(\bm{x}_{i}),(j^{\prime},j^{\prime\prime})\in\bm{C}_{i}^{+}\times\bm{C}_{i}^{-}\}$. Then,

[TABLE] 2. 2.

Average AUC (Auc): This is the fraction that a positive instance is ranked higher than a negative instance, averaged over all labels. Specifically, for label $j$, define $\bm{\tilde{Q}}_{j}=\{(i^{\prime},i^{\prime\prime})\;|\;f_{j}(\bm{x}_{i^{\prime}})\geq f_{j}(\bm{x}_{i^{\prime\prime}}),(\bm{x}_{i^{\prime}},\bm{x}_{i^{\prime\prime}})\in\bm{Z}_{j}^{+}\times\bm{Z}_{j}^{-}\}$. Then,

[TABLE] 3. 3.

Coverage (Cvg): This counts how many steps are needed to move down the predicted label ranking so as to cover all the positive labels of the instances.

[TABLE] 4. 4.

Average precision (Ap): This is the average fraction of positive labels ranked higher than a particular positive label. For instance $i$, define $\bm{\hat{Q}}_{i,c}=\{j\;|\;\mathrm{rank}_{\bm{f}}(\bm{x}_{i},j)\leq\mathrm{rank}_{\bm{f}}(\bm{x}_{i},c),j\in\bm{C}_{i}^{+}\}$. Then,

[TABLE]

For Auc and Ap, the higher the better; whereas for Rkl and Cvg, the lower the better. To reduce statistical variability, results are averaged over 10 independent repetitions.

4.2 Learning with Full Labels

In this experiment, all elements in the training label matrix are observed. Performance on the test data is shown in Table 2. As expected, BR is the worst , since it treats each label independently without considering label correlations. MLLOC only considers local label correlations and LEML only makes use of the low-rank structure. Though ML-LRC takes advantage of both the low-rank structure and label correlations, only global label correlations are considered. As a result, GLOCAL is the best overall, as it models both global and local label correlations.

To show the example correlations learned by GLOCAL, we use two local groups extracted from the Image dataset. Figure 1 shows that local label correlation does vary from group to group, and is different from global correlation. For group 1, “sunset” is highly correlated with “desert” and “sea” (Figure 1(c)). This can also be seen from the images in Figure 1(a). Moreover, “trees” sometimes co-occurs with “deserts” (first and last images in Figure 1(a)). However, in group 2 (Figure 1(d)), “mountain” and “sea” often occur together and “trees” occurs less often with “desert” (Figure 1(b)). Figure 1(e) shows the learned global label correlation: “sea” and “sunset”, “mountain” and “trees” are positively correlated, whereas “desert” and “sea”, “desert” and “trees” are negatively correlated. All these correlations are consistent with intuition.

To further validate the effectiveness of global and local label correlations, we study two degenerate versions of GLOCAL: (i) GLObal, which uses only global label correlations; and (ii) loCAL, which uses only local label correlations. Note that the local groups obtained by clustering are not of equal sizes. For some datasets, the largest cluster contains more than $40\%$ of instances, while some small ones contain fewer than $5\%$ each. Global correlation is then dominated by the local correlation matrix of the largest cluster (Proposition 1), making the performance difference on the whole test set obscure. Hence, we focus on the performance of the small clusters. As can be seen from Table 3, using only global or local correlation may be good enough on some data sets (such as Health). On the other hand, considering both types of correlation as in GLOCAL achieves comparable or even better performance.

4.3 Learning with Missing Labels

In this experiment, we randomly sample $\rho\%$ of the elements in the label matrix as observed, and the rest as missing. Note that BR and MLLOC can only handle datasets with full labels. Hence, we first use MAXIDE xu2013speedup , a matrix completion algorithm for transductive multi-label learning, to fill in the missing labels before they can be applied. We use MBR for MAXIDE+BR, and MMLLOC for MAXIDE+MLLOC.

Tables 4 and 5 show the results on the training and test data, respectively.555To fit the tables on one page, we do not report the standard deviation. MBR, which performs worst, is also not shown.

As can be seen, performance increases with more observed entries in general. Overall, GLOCAL performs best at different $\rho$’s, as it simultaneously considers both global and local label correlations with label manifold regularization. In contrast, MBR and MMLLOC handle label recovery and learning separately. Moreover, MMLLOC takes only local label correlation, and MBR does not consider label correlations. As a result, they perform much worse than GLOCAL. Though LEML and ML-LRC perform learning with missing label recovery together, they consider only global correlation, and are thus often worse than GLOCAL.

4.4 Convergence

In this section, we empirically study the convergence of GLOCAL. Figure 2 shows the objective value w.r.t. the number of iterations for the full-label case. Because of the lack of space, results are only shown on the Arts, Business, Enron and Image datasets. As can be seen, the objective converges quickly in a few iterations. A similar phenomenon can be observed on the other datasets.

Table 6 shows the timing results on learning with missing labels (with $\rho=70$). GLOCAL and LEML train a classifier for all the labels jointly, and also can take advantage of the low-rank structure of either the model or label matrix during training. Thus, they are the fastest. However, GLOCAL has to be warm-started by Eqn. (1), and requires an additional clustering step to obtain local groups of the instances. Hence, it is slower than LEML. However, as have been observed in previous sections, GLOCAL outperforms LEML in terms of label recovery. ML-LRC uses a low-rank label correlation matrix. However, it does not reduce the size of the label matrix or model involved in each iteration, and so is slower than GLOCAL. MBR and MMLLOC require training a classifier for each label, and also an additional step to recover the missing labels. Thus, they are often the slowest, especially when the number of class labels is large. Similar results can be observed with $\rho=30$, which are not reported here.

4.5 Sensitivity to Parameters

In this experiment, we study the influence of parameters, including the number of clusters $g$, regularization parameters $\lambda_{3}$ and $\lambda_{4}$ (corresponding to the manifold regularizer for global and local label correlations, respectively), regularization parameter $\lambda_{2}$ for the Frobenius norm regularizer, and dimensionality $k$ of the latent representation. We vary one parameter, while keeping the others fixed at their best setting.

4.5.1 Varying the Number of Clusters $g$

Figure 3 shows the influence on the Enron dataset. When there is only one cluster, no local label correlation is considered. With more clusters, performance improves as more local label correlations are taken into account. When too many clusters are used, very few instances are placed in each cluster, and the local label correlations cannot be reliably estimated. Thus, the performance starts to deteriorate.

4.5.2 Influence of Label Manifold Regularizers ($\lambda_{3}$ and $\lambda_{4}$)

A larger $\lambda_{3}$ means higher importance of global label correlation, whereas a larger $\lambda_{4}$ means higher importance of local label correlation. Figures 5 and 5 show their effects on the Enron dataset. When $\lambda_{3}=0$, only local label correlations are considered, and the performance is poor. With increasing $\lambda_{3}$, performance improves. However, when $\lambda_{3}$ is very large, performance deteriorates as the global label correlations dominate. A similar phenomenon can be observed for $\lambda_{4}$.

4.5.3 Varying the Latent Representation Dimensionality $k$

Figure 6 shows the effect of varying $k$ on the Enron dataset. As can be seen, when $k$ is too small, the latent representation cannot capture enough information. With increasing $k$, performance improves. When $k$ is too large, the low-rank structure is not fully utilized, and performance starts to get worse.

4.5.4 Influence of $\lambda_{2}$

Figure 7 shows the effect of varying $\lambda_{2}$ on the Enron dataset. As can be seen, GLOCAL is not sensitive to this parameter.

5 Conclusion

In this paper, we proposed a new multi-label correlation learning approach GLOCAL, which simultaneously recovers the missing labels, trains the classifier and exploits both global and local label correlations, through learning a latent label representation and optimizing the label manifolds. Compared with the previous work, it is the first to exploit both global and local label correlations, which directly learns the Laplacian matrix without requiring any other prior knowledge on label correlations. As a result, the classifier outputs and label correlations best match each other, both globally and locally. Moreover, GLOCAL provides a unified solution for both full-label and missing-label multi-label learning. Experimental results show that our approach outperforms the state-of-the-art multi-label learning approaches on learning with both full labels and missing labels. In our work, we handle the case that label correlations are symmetric. In many situations, correlations can be asymmetric. For example, “mountain” are highly correlated to “tree”, since it is very common that a mountain has trees in it. However, “tree” may be less correlated to “mountain”, because trees can be found not only in mountains, but often in the streets, parks, etc. So it is desirable to study the asymmetric label correlations in our future work.

Acknowledgment

This research was supported by NSFC (61333014), 111 Project (B14020), and the Collaborative Innovation Center of Novel Software Technology and Industrialization.