Self-Weighted Multiview Metric Learning by Maximizing the Cross Correlations

TL;DR

This paper introduces SM2L, a multiview metric learning algorithm that maximizes cross-view correlations and adaptively weights different views, improving similarity measurement in multimedia applications.

Contribution

The paper presents a novel self-weighted multiview metric learning method that maximizes cross correlations and assigns adaptive weights to views, enhancing multiview data analysis.

Findings

SM2L outperforms existing methods on benchmark datasets.

Adaptive view weighting improves learning performance.

Maximizing cross correlations effectively captures multiview relationships.

Abstract

With the development of multimedia time, one sample can always be described from multiple views which contain compatible and complementary information. Most algorithms cannot take information from multiple views into considerations and fail to achieve desirable performance in most situations. For many applications, such as image retrieval, face recognition, etc., an appropriate distance metric can better reflect the similarities between various samples. Therefore, how to construct a good distance metric learning methods which can deal with multiview data has been an important topic during the last decade. In this paper, we proposed a novel algorithm named Self-weighted Multiview Metric Learning (SM2L) which can finish this task by maximizing the cross correlations between different views. Furthermore, because multiple views have different contributions to the learning procedure of SM2L, we adopt a self-weighted learning framework to assign multiple views with different weights. Various experiments on benchmark datasets can verify the performance of our proposed method.

Figures

Tables

Table 1. Table 1 : The classification accuracies ( % ) \left({\%}\right) on 3Sources dataset

Methods	120		80
	Mean	Max	Mean	Max
ITML	82.13	89.44	76.38	83.23
CMM	61.29	66.01	59.33	64.52
LMNN	73.97	84.63	70.76	80.95
MML	83.32	92.01	78.83	89.32
SM2L	85.07	92.71	79.93	90.32

Table 2. Table 2 : The classification accuracies ( % ) \left({\%}\right) on Cora dataset

Methods	1900		1400
	Mean	Max	Mean	Max
ITML	60.89	62.01	60.15	61.76
CMM	65.55	67.03	62.39	63.44
LMNN	66.58	68.14	63.26	64.89
MML	67.14	68.32	64.22	66.32
SM2L	67.44	68.59	65.23	66.93

Table 3. Table 3 : The classification accuracies ( % ) \left({\%}\right) on WebKB dataset

Subsets		ITML	CMM	LMNN	MML	SM2L
WebKB1	Mean	84.23	83.10	84.93	85.07	86.12
	Max	86.88	88.83	88.12	90.71	91.27
WebKB2	Mean	82.04	81.14	81.32	81.93	82.58
	Max	91.01	88.07	91.15	91.32	91.66
WebKB3	Mean	90.55	89.78	90.33	90.52	90.92
	Max	93.61	94.05	94.11	92.67	93.87
WebKB4	Mean	84.63	82.78	82.01	85.65	86.54
	Max	88.22	90.54	88.44	91.35	92.33

Equations

$$\begin{array}[]{l}d_{\bm{A}^{v}}\left(\bm{x}^{v}_{i},\bm{x}^{v}_{j}\right)=\sqrt{\left(\bm{x}^{v}_{i}-\bm{x}^{v}_{j}\right)^{T}\bm{A}^{v}\left(\bm{x}^{v}_{i}-\bm{x}^{v}_{j}\right)}\end{array}$$

$$\begin{array}[]{l}\mathop{\arg\min}\limits_{\bm{\alpha},\bm{Y}}\sum\limits_{v=1}^{m}{\alpha_{v}tr\left({\bm{Y}\bm{L}^{\left(v\right)}\bm{Y}^{T}}\right)}\\ s.t.\;\bm{Y}\bm{Y}^{T}=\bm{I};\;\sum\limits_{v=1}^{m}{\alpha_{v}}=1,\;\;\alpha_{v}\geq 0\\ \end{array}$$

$$\begin{array}[]{l}\mathop{\arg\max}\limits_{\bm{P}^{1},\bm{P}^{2},\cdots,\bm{P}^{m}}\sum\limits_{i=1}^{m}\alpha_{i}tr\left((\bm{P}^{i})^{T}\bm{S}_{b}^{i}(\bm{P}^{i})\right)\\ ~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}+\sum\limits_{i<j}\lambda_{i}jtr\left((\bm{P}^{i})^{T}\bm{X}^{i}(\bm{X}^{j})^{T}\bm{P}^{j}\right)\\ ~{}~{}~{}s.t.\sum\limits_{i=1}^{m}\gamma_{i}tr\left((\bm{P}^{i})^{T}\bm{S}_{w}^{i}(\bm{P}^{i})\right)=\bm{I}_{p}\\ \end{array}$$

$$\begin{array}[]{l}\mathop{\arg\max}\limits_{\bm{A}^{1},\bm{A}^{2},\cdots,\bm{A}^{m}}\sum\limits_{v=1}^{m}\left\{\sum\limits_{i,j:(\bm{x}^{v}_{i},\bm{x}^{v}_{j})\in D}\frac{d^{2}_{\bm{A}^{v}}(\bm{x}^{v}_{i},\bm{x}^{v}_{j})}{N_{D}}\right.\\ ~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\left.-\sum\limits_{i,j:(\bm{x}^{v}_{i},\bm{x}^{v}_{j})\in S}\frac{d^{2}_{\bm{A}^{v}}(\bm{x}^{v}_{i},\bm{x}^{v}_{j})}{N_{S}}\right\}\\ \end{array}$$

$$\begin{array}[]{l}\mathop{\arg\max}\limits_{\bm{W}^{1},\cdots,\bm{W}^{m}}\sum\limits_{v=1}^{m}\par tr\left\{(\bm{W}^{v})^{T}\sum\limits_{(\bm{x}^{v}_{i},\bm{x}^{v}_{j})\in D}\frac{\left(\bm{x}^{v}_{i}-\bm{x}^{v}_{j}\right)\left(\bm{x}^{v}_{i}-\bm{x}^{v}_{j}\right)^{T}}{N_{D}}\bm{W}^{v}\right.\\ ~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\left.-(\bm{W}^{v})^{T}\sum\limits_{(\bm{x}^{v}_{q},\bm{x}^{v}_{p})\in S}\frac{\left(\bm{x}^{v}_{q}-\bm{x}^{v}_{p}\right)\left(\bm{x}^{v}_{q}-\bm{x}^{v}_{p}\right)^{T}}{N_{S}}\bm{W}^{v}\right\}\par\end{array}$$

$$\begin{array}[]{l}\mathop{\arg\max}\limits_{\bm{W}^{1},\bm{W}^{2},\cdots,\bm{W}^{m}}\sum\limits_{v=1}^{m}tr\left\{(\bm{W}^{v})^{T}\left(\bm{M}_{D}-\bm{M}_{S}\right)\bm{W}^{v}\right\}\\ s.t.(\bm{W}^{v})^{T}\bm{W}^{v}=\bm{I},v=1,2,\cdots,m\end{array}$$

$$\begin{array}[]{l}\mathop{\arg\max}\limits_{\bm{W}^{1},\bm{W}^{2},\cdots,\bm{W}^{m}}\sum\limits_{v=1}^{m}tr\left\{(\bm{W}^{v})^{T}\left(\bm{M}_{D}-\bm{M}_{S}\right)\bm{W}^{v}\right\}\\ ~{}~{}~{}~{}~{}~{}+\sum\limits_{v\neq w}\lambda_{ij}tr\left((\bm{W}^{v})^{T}\bm{X}^{v}(\bm{X}^{w})^{T}\bm{W}^{w}\right)\\ s.t.(\bm{W}^{v})^{T}\bm{W}^{v}=\bm{I},v=1,2,\cdots,m\end{array}$$

$$\begin{array}[]{l}\max\mathcal{G}\left(\bm{\alpha},\bm{W}^{1},\bm{W}^{2},\cdots,\bm{W}^{m}\right)\\ ~{}~{}~{}~{}~{}~{}=\sum\limits_{v=1}^{m}\alpha^{r}_{v}tr\left((\bm{W}^{v})^{T}\left(\bm{M}_{D}-\bm{M}_{S}\right)\bm{W}^{v}\right)\\ ~{}~{}~{}~{}~{}~{}+\sum\limits_{v\neq w}\frac{\alpha^{r}_{v}+\alpha^{r}_{w}}{2\eta}tr\left((\bm{W}^{v})^{T}\bm{X}^{v}(\bm{X}^{w})^{T}\bm{W}^{w}\right)\\ s.t.(\bm{W}^{v})^{T}\bm{W}^{v}=\bm{I},v=1,2,\cdots,m\\ ~{}~{}~{}~{}~{}~{}\sum\limits_{v=1}^{m}\alpha_{v}=1,r>1\end{array}$$

$$\mathcal{L}\left(\mathbf{\alpha },\lambda \right)=\mathcal{G}\left(\mathbf{\alpha },{\mathbf{W}}^1,{\mathbf{W}}^2,\cdots ,{\mathbf{W}}^m\right)-\lambda \left(\sum _{v=1}^m{\alpha }_v-1\right)$$

$$\{\genfrac{}{}{0ex}{}{\frac{\partial \mathcal{L}\left(\mathbf{\alpha },\lambda \right)}{\partial {\alpha }_v}=\frac{\partial \mathcal{G}\left(\mathbf{\alpha },{\mathbf{W}}^1,{\mathbf{W}}^2,\cdots ,{\mathbf{W}}^m\right)}{\partial {\alpha }_v}-\lambda =0,v=1,2,\cdots ,m}{\frac{\partial \mathcal{L}\left(\mathbf{\alpha },\lambda \right)}{\partial \lambda }=\sum _{v=1}^m{\alpha }_v-1=0}$$

$${\alpha }_v=\frac{{\left(1/tr\left({\left({\mathbf{U}}^v\right)}^T{\mathcal{J}}^v{\mathbf{U}}^v\right)\right)}^{1/\left(r-1\right)}}{\sum _{v=1}^m{\left(1/tr\left({\left({\mathbf{U}}^v\right)}^T{\mathcal{J}}^v{\mathbf{U}}^v\right)\right)}^{1/\left(r-1\right)}}$$

$$\begin{array}[]{l}\mathcal{J}^{v}=tr\left((\bm{W}^{v})^{T}\left(\bm{M}_{D}-\bm{M}_{S}\right)\bm{W}^{v}\right)\\ ~{}~{}~{}~{}~{}~{}+\frac{1}{2\eta}tr\left((\bm{W}^{v})^{T}\bm{X}^{v}(\bm{X}^{w})^{T}\bm{W}^{w}\right)\end{array}$$

Full text

In this paper, we proposed a novel algorithm named Self-weighted Multiview Metric Learning (SM2L) which can finish this task by maximizing the cross correlations between different views. Furthermore, because multiple views have different contributions to the learning procedure of SM2L, we adopt a self-weighted learning framework to assign multiple views with different weights. Various experiments on benchmark datasets can verify the performance of our proposed method.

**Index Terms— ** Multiview learning, Metric learning, Self-weighted

1 Introduction

With the coming of information period, massive data has been generated from various fields to meet the requirement of different people. It is a common phenomena that one same sample can always be described by different tools (or techniques). For example, an image can be represented by several descriptors, such as Local Binary Patterns (LBP) [1], Histogram of Oriented Gradient (Hog) [2] and Locality-constrained Linear Coding (LLC) [3]. Because these descriptors contain different which is compatible and complementary, how to exploit the multiview data to further improve the performance of various applications [4, 5, 6] is of vital importance but challenging.

For many applications, such as image retrieval [7], face recognition [8], etc. [9, 10], learning an appropriate distance metric can help these applications to measure the similarities between samples precisely. Xing et al.[11] proposed a convex optimization function, which minimizes the distances between samples from the same class. Meanwhile, they introduced two constraints to obtain a better metric. Weinberger et al. [12] introduced a DML method named Large Margin Nearest Neighbor (LMNN), which refers the thoughts of SMVs [13]. Information-Theoretic Metric Learning (ITML) [14] is another well-known DML methods which combimes information theory [15] with distance metric learning (DML) and achieves good performances. Neighborhood components analysis (NCA) [16] trains a distance metric by maximizes the probabilities of that samples and their stochastic neighbors are from the same class. Even though these algorhtms have wide influences on this field, most DML methods cannot deal with multiview data simultaneously and waste a lot of important information.

In order to deal with the problem above, multiview learning [17, 18] has been studied recently. Kumar et al. [19] developed a co-regularized framework to minimize the distinctons between all views. All views are forced to learn from each other via the proposed framework. Xia et al.[20] extended spectral embedding into multiview mode and proposed a self-weighted method to calcuate the weights of all views. Canonical correlation analysis (CCA) [21] is another good multiview method which is an extension of Principal component analysis (PCA) [22] and constructs the low-dimensional subspace by maximizing the cross correlations of each two views. Zhai et al. [23] combined global consistency with local smoothness and obtained a multiview metric learning method. Futhermore, some dimension reduction techniques can also be utilized as DML methods, such as PCA [22] and LDA [24] .Till now, most distance metric leanring methdos cannot deal with multiview data and obtain better performances. It is essential for researchers to conduct more studies on this field [25].

In this paper, we propose a novel multiview metric learning methods named Self-weighted Multiview Metric Learning (SM2L) which can fully exploit multiview data to measure similarities between differetn samples. The proposed SM2L first introduced the maximum margin criterion for all views to maximize the distances between samples from different classes in each view. Then, in order to force all views to learn from each other, SM2L aims to maximize the cross correlations between the multiview features of one same sample. Finally, because multiple views have different impacts on the construction process of distance metric, SM2L equips multiple views with different weights which can be learned automatically.

This paper is organized as follows: in section 2, we introduced some basic knowledge and some related works. In section 3, we introduced the construction procedure of our proposed SM2L in detail and the solving procedure of it. In section 4, we conducted several experiments to show the excellent performance of our proposed method. And section 5 made a conclusion in detail.

2 Related works

In this section, we first introduced some basic knowlege about multiview distance metric learning. Then, we introduced 2 related algorithms.

2.1 Basic Knowledge

Assume we are given a set of multiview data $\bm{X}=\left\{\bm{X}^{v}\in\Re^{D_{v}\times n},v=1,\cdots,m\right\}$ which consists of $n$ samples from $m$ views, where $\bm{X}^{v}=\left\{\bm{x}^{v}_{1},\bm{x}^{v}_{2},\cdots,\bm{x}^{v}_{n}\right\}\in\Re^{D_{v}\times n}$ contains all features from the $v$th view. For any two features $\bm{x}^{v}_{i}$ and $\bm{x}^{v}_{j}$ in the $v$th view, their Mahalanobis distance can be computed as:

[TABLE]

Where $\bm{A}^{v}$ is the distance metric matrix for the $v$th view. $d_{\bm{A}^{v}}\left(\bm{x}^{v}_{i},\bm{x}^{v}_{j}\right)$ represents the distance between $\bm{x}^{v}_{i}$ and $\bm{x}^{v}_{j}$. Because $\bm{A}^{v}$ is a semi-definite matrix, it can be be decomposed into $\bm{A}^{v}=\bm{W}^{v}\left(\bm{W}^{v}\right)^{T}$, $\bm{W}^{v}\in\Re^{d\times D_{v}}$ and $d_{v}\leq D_{v}$ using eigenvalue decomposition.

For multiview distance metric learning methods, it’s essential for all views to learn from each other to improve the quality of distance metric for each view. And for any distance metric, it should satisfy these four properties as follows:

1. triangular inequality: $d_{\bm{A}^{v}}\left(\bm{x}^{v}_{i},\bm{x}^{v}_{j}\right)+d_{\bm{A}^{v}}\left(\bm{x}^{v}_{j},\bm{x}^{v}_{k}\right)\geq d_{\bm{A}^{v}}\left(\bm{x}^{v}_{i},\bm{x}^{v}_{k}\right)$;

2. non-negativity: $d_{\bm{A}^{v}}\left(\bm{x}^{v}_{i},\bm{x}^{v}_{j}\right)\geq 0$;

3. symmetry: $d_{\bm{A}^{v}}\left(\bm{x}^{v}_{i},\bm{x}^{v}_{j}\right)=d_{\bm{A}^{v}}\left(\bm{x}^{v}_{j},\bm{x}^{v}_{i}\right)$;

4. distinguishability: $d_{\bm{A}^{v}}\left(\bm{x}^{v}_{i},\bm{x}^{v}_{j}\right)=0\leftrightarrow\bm{x}^{v}_{i}=\bm{x}^{v}_{j}$;

2.2 Related Algorithms

In this section, we introduced some related algorithms which have been refered in our proposed SM2L.

2.2.1 Multiview Spectral Embedding

Multiview Spectral Embedding (MSE) [20] is a well-known multi-view dimension reduction method which can assign all views with appropriate weights automatically. It encodes different features from multiple views to achieve a physically meaningful embedding. Furthermore, MSE integrates laplacian graphs from multiple views via global coordinate alignment, which help all views to learn from each other. And the objective function of MSE has been shown as follows:

[TABLE]

$\bm{\alpha}=\left[{\alpha_{1},\alpha_{2},\cdots,\alpha_{m}}\right]$ is a set of coefficients which can reflect the importance of different views. $\bm{L}^{\left(v\right)}$ is the laplacian graph for features in the $v$th view. It reflects the neighborhood relationship between features in the $v$th view. And $\bm{Y}$ is the commen low-dimensional representation for the original multi-view data. In order to obtain the optimal $\bm{Y}$, MSE develops an iterative optimization procedure to update $\bm{\alpha}$ and $\bm{Y}$ alternately. However, MSE is not a multiview metric learning method which cannot measure similarities between multiview data.

2.2.2 Generalized Multi-view Analysis

Sharma et al. [26] proposed a multiview learning method to construct subspace for the task of classification [27]. They extended linear discriminate analysis (LDA) [24] into multiview mode by maximizing the cross correlations between each two views as follows:

[TABLE]

Where $\bm{P}_{i}$ represents the projection matrix for features in the $i$th view. $\bm{S}_{b}^{i}$ is the between-class scatter matrix while $\bm{S}_{w}^{i}$ is the with-in scatter one. $\alpha_{i}$ and $\gamma_{i}$ are the weights for between and with-in class degree the $i$th view. And $\lambda_{i}j$ is the regularized parameter. Even though Eq.3 maximize the the cross correlations between each two views, they cannot assign all views with different weights automatically, which causes too many parameters need to be set during the experiments.

3 The Proposed SM2L

In this paper, we describe the construction procedure of SM2L in detail as fig.1. SM2L first adopts maximize maximum margin criterion to help all views to construct distance metric matrix by maximize the distances between features from different classes. Then, SM2L forces all views to learn from each other by maximizing the cross correlations between each two views. Finally, SM2L assigns all views with different weights automatically to fully exploit the infromation of multiview data.

3.1 The Construction Procedure of SM2L

For multiview data $\bm{X}=\left\{\bm{X}^{v}\in\Re^{D_{v}\times n},v=1,\cdots,m\right\}$ together with pairwise constraints $S$ and $D$ as follows:

$S:\forall(\bm{x}^{v}_{i},\bm{x}^{v}_{j})\in S$, $~{}~{}~{}~{}~{}\bm{x}^{v}_{i},\bm{x}^{v}_{j}\in$ same class

$D:\forall(\bm{x}^{v}_{i},\bm{x}^{v}_{j})\in D$, $~{}~{}~{}~{}\bm{x}^{v}_{i},\bm{x}^{v}_{j}\in$ different classes

Then, according to the constraints above, SM2L aims to maximize the distances between features from differetn classes while minimizing the distances between features in the same class as follows:

[TABLE]

Where $N_{S}$ and $N_{D}$ are the numbers of pairwise constraints in $S$ and $D$. It is clearly that Eq.4 maximizes the distances between features from the different classes for all views. Due to Eq.1 and $\bm{A}^{v}=\bm{W}^{v}\left(\bm{W}^{v}\right)^{T}$, Eq.4 can be further transformed as follows:

[TABLE]

And Eq.5 equals to

[TABLE]

Where $M_{D}=\sum\limits_{(\bm{x}^{v}_{i},\bm{x}^{v}_{j})\in D}\frac{\left(\bm{x}^{v}_{i}-\bm{x}^{v}_{j}\right)\left(\bm{x}^{v}_{i}-\bm{x}^{v}_{j}\right)^{T}}{N_{D}}$ and $M_{S}=\sum\limits_{(\bm{x}^{v}_{i},\bm{x}^{v}_{j})\in S}\frac{\left(\bm{x}^{v}_{i}-\bm{x}^{v}_{j}\right)\left(\bm{x}^{v}_{i}-\bm{x}^{v}_{j}\right)^{T}}{N_{S}}$. Therefore, the construction of $\bm{A}^{v}$ is equal to find the optimal $\bm{W}^{v}$.

However, Eq.5 cannot integrate infromation from multiple views. In order to fully take multiview data into consideration, SM2L adopts the idea from [26] and maximizes the cross correlations betwee each 2 views as follows:

[TABLE]

Through Eq.7, all views can learn from each other to further improve the performance of distance metric learning. However, because multiple views have different impacts on the construction of their distance metric matrix, it is essential for SM2L to learn the weights of these views automatically. Therefore, the objective function of SM2L can be organized as follows:

[TABLE]

Where $\bm{\alpha}=\left[\alpha_{1},\alpha_{2},\cdots,\alpha_{m}\right]$ consists of weights of multiple views. $r>1$ ensures that all views have impacts on the construction of distance metric matrix. Therefore, once the optimal $\bm{W}^{v}$ is obtained for the $v$th view, the distance metric matrix can be calculated by $\bm{A}^{v}=\bm{W}^{v}\left(\bm{W}^{v}\right)^{T}$.

3.2 The Solving Procedure of SM2L

In this section, we introduce the solving procedure of SM2L in detail. Because it is difficult to obtain $\bm{\alpha}$ and $\bm{W}^{1},\bm{W}^{2},\cdots,\bm{W}^{m}$ simultaneously. SM2L adopts alternately iteration to update the variables separately.

Update $\bm{W}^{1},\bm{W}^{2},\cdots,\bm{W}^{m}$: SM2L keeps $\bm{\alpha}$ unchanged to update $\bm{W}^{v},v=1,2,\cdots,m$ solely. It has been verified in [26] that the optimization procedure for $\bm{W}^{1},\bm{W}^{2},\cdots,\bm{W}^{m}$ is a genelarized eigenvalue decomposition problem. The solving procedure of SM2 is similar with that in [26].

Update $\bm{\alpha}$: After $\bm{W}^{1},\bm{W}^{2},\cdots,\bm{W}^{m}$ are obtained, how to calculate $\bm{\alpha}$ is a main problem. By using a Lagrange multiplier $\lambda$ to take the constraint $\sum_{v=1}^{m}\alpha_{v}=1$ into consideration, we get the Lagrange function as:

[TABLE]

By setting the derivative of $\mathcal{L}\left(\bm{\alpha},\lambda\right)$ with respect to $\alpha_{v}$ and $\lambda$ to zero, we can get:

[TABLE]

Through Eq.9, we can get $\alpha_{v}$:

[TABLE]

where

[TABLE]

And we can update $\alpha_{v},v=1,2,\cdots,m$ according to Eq.11. SM2L iterative $\bm{\alpha}$ and $\bm{W}^{1},\bm{W}^{2},\cdots,\bm{W}^{m}$ alternately. And the iteration stops once the obtained $\bm{W}^{1},\bm{W}^{2},\cdots,\bm{W}^{m}$ converge.

4 Experiment

In this section, we conducted various experiments on benchmark datasets. And the experiments have verified that our proposed SM2L can achieve good performances.

4.1 Datasets and Comparing Methods

In this section, we introduced the utilized datasets and the comparing methdos in detail. There are 3 datasets utilized in our experiments, including 3Sources 111http://mlg.ucd.ie/datasets/3sources.html, Cora 222http://lig-membres.imag.fr/grimal/data.html, WebKB 333http://www.webkb.org. All these datasets are benchmark multiview datasets.

3Sources is a benchmark multiview dataset and consists of 3 well-known online news sources: BBC, Reuters and the Guardian. Each source was utilized as one single view and this dataset selected 169 stories as samples. Cora contains 2708 publications which come from 7 classes. Contents and cites are utilized as 2 views in Cora dataset. WebKB contains 4 subsets of documents over 6 categories. All samples in WebKB consists features from 3 views in one page, including the text on it, the anchor text on the hyperlink pointing to it and the text in its title.

In order to show the excellent performance of SM2L, we compare it with several distance metric learning methods in the experiments, including ITML [14], CMM [28], LMNN [12] and MML [29]. ITML, CMM and LMNN are 3 famous single view DML methods. They cannot utilized features from multiple views at the same time. Therefore, we trained them on these views and shown the best performances. Both MML and our proposed SM2L are multiview DML methods.

4.2 Experiment Results

In this section, we conducted experiments on the benchmark datasets which has been described above and shown the results carefully. In our experiments, all methods are trained or tested using the same training or testing samples.

For 3Sources dataset, our experiment randomly select 120 and 80 samples as training ones while the other ones are assigned as testing ones. All DML methods trained their distance metric matrix after the training phase, the experiment adopts 1NN classification on the testing samples. All the DML methods are conducted 10 times and the mean and max classification accuracies are shown as Table 2.

It is clearly that SM2L can achive best performances on 3Sources dataset. As single view DML methods, ITML, CMM and LMNN cannot achieve satisfactory performances. Because MML and SM2L are multiview metric learning methods, they are better than those 3 single view ones. Because multiview algorithms take more information into considerations, they are better in most situations.

For Cora dataset, our experiment randomly select 1900 and 1400 samples as training ones while the other ones are assigned as testing ones. All DML methods trained their distance metric matrix after the training phase, the experiment adopts 1NN classification on the testing samples. All the DML methods are conducted 10 times and the mean and max classification accuracies are shown as Table 2.

SM2L can achieve best performances on Cora dataset. Meanwhile, MML is another good distance metric learning for multiview data. Fpr those 3 single view methods, LMNN and CMM are better than ITML.

For 4 subsets of WebKB, we randomly select $70\%$ samples as training ones while the other(about $30\%$) are selected as testing ones. All DML methods trained their distance metric matrix after the training phase, the experiment adopts 1NN classification on the testing samples. All the DML methods are conducted 10 times and the mean and max classification accuracies are shown as Table 3.

It can be found in Table 3 that SM2L can achieve best performances in most situations. For 4 subsets of WebKB, because MML and SM2L are multiview methods which can full exploit multiview data, they are better than those 3 single view methods. For those 3 single view methods, LMNN is the best one for WebKB.

5 Conclusion

In this paper, we propose a novel multiview distance metric learning method named Self-weighted Multiview Metric Learning (SM2L). The proposed SM2L utilized maximum margin criterion to maximize the distances between multiview features from different classes while minimize the distances between multiview features from the same ones. Furthermore, SM2L integrates information from all views maximizing the cross correlations between each 2 views. Finally, we show the solving procedure of SM2L in detail. The experiment results have shown the superiority of SM2L.