Ordinal Distribution Regression for Gait-based Age Estimation

TL;DR

This paper introduces an ordinal distribution regression approach using a global and local CNN for gait-based age estimation, effectively capturing age ordinal relationships and improving accuracy over existing methods.

Contribution

It proposes a novel ordinal distribution regression framework with a combined global and local CNN architecture for more accurate gait-based age estimation.

Findings

Outperforms state-of-the-art methods on OULP-Age dataset

Effectively models ordinal age relationships

Utilizes global and local features for improved accuracy

Abstract

Computer vision researchers prefer to estimate age from face images because facial features provide useful information. However, estimating age from face images becomes challenging when people are distant from the camera or occluded. A person's gait is a unique biometric feature that can be perceived efficiently even at a distance. Thus, gait can be used to predict age when face images are not available. However, existing gait-based classification or regression methods ignore the ordinal relationship of different ages, which is an important clue for age estimation. This paper proposes an ordinal distribution regression with a global and local convolutional neural network for gait-based age estimation. Specifically, we decompose gait-based age regression into a series of binary classifications to incorporate the ordinal age information. Then, an ordinal distribution loss is proposed to consider the inner relationships among these classifications by penalizing the distribution discrepancy between the estimated value and the ground truth. In addition, our neural network comprises a global and three local sub-networks, and thus, is capable of learning the global structure and local details from the head, body, and feet. Experimental results indicate that the proposed approach outperforms state-of-the-art gait-based age estimation methods on the OULP-Age dataset.

Tables

Table 1. Table 1: Comparison of the age estimation MAEs with the proposed approach and state-of-the-art methods on the OULP-Age dataset.

Methods	MAE	CS ($k=5$)
SVR [34]	7.66	41.40%
MLG [15]	10.98	43.40%
OPLDA [16]	8.45	36.50%
OPMFA [16]	9.08	34.70%
GPR [21]	7.30	43.60%
ASSOLPP [14]	6.78	53.00%
VGG16 + Mean-Variance [23]	5.59	60.46%
ODR-GLCNN (Ours)	5.12	66.95%

Table 2. Table 2: MAE results on the two subsets of the USF dataset using different methods.

Methods	Gallery		Probe A
	MAE	CS($k=5$)	MAE	CS($k=5$)
SVR [34]	8.21	37.50%	7.83	41.70%
MLG [15]	9.45	32.80%	9.06	34.40%
OPLDA [16]	7.05	43.70%	6.76	51.20%
OPMFA [16]	6.95	47.30%	6.62	52.00%
ASSOLPP [14]	6.81	50.50%	6.48	52.70%
VGG16 + Mean-Variance [23]	6.10	54.20%	5.93	56.30%
ODR-GLCNN (Ours)	5.91	56.60%	5.75	59.40%

Table 3. Table 3: Comparisons among different CNN-based methods on the OULP-Age dataset. The test time for a single sample is shown in the last column.

Network	MAE	CS ($k=5$)	Time (ms)
CNN	5.45	64.64%	7.27 $\times {10}^{-2}$
VGG16	5.63	63.92%	21.9 $\times {10}^{-2}$
GL-CNN (Ours)	5.24	65.96%	8.99 $\times {10}^{-2}$

Table 4. Table 4: Comparison of different losses with the proposed GL-CNN on the OULP-Age dataset.

Loss	MAE	CS ($k=5$)
Euclidean	6.73	52.95%
MAE	6.65	55.16%
Squared EMD[10]	6.39	58.34%
Cross-Entropy	5.24	65.96%
ODL (Ours)	5.12	66.95%

Table 5. Table 5: Influence of human gender for gait-based age estimation in terms of age MAE and gender accuracy.

Method	w/o gender	w/ gender	acc.
CNN + Euclidean	6.96	6.82	96.70%
CNN + Cross-Entropy	5.40	5.34	97.20%
VGG16 + Mean-Variance	5.59	5.52	96.70%
ODR-GLCNN (Ours)	5.12	5.06	97.80%

Table 6. Table 6: Comparison between proposed approach and state-of-the-art methods on the MORPH Album II dataset in terms of MAE and CS values (*: the value is read from the reported CS curve). RS represents the five-fold R andom S plit protocol.

Method	MAE	CS ($k=5$)	Protocol
OR-CNN [22]	3.27	73.0%*	RS
DEX [27]	3.25	–	RS
Ranking-CNN [3]	2.96	85.0%*	RS
VGG16 + Mean-Variance [23]	2.41	90.0%*	RS
AGEn [43]	2.93	–	RS
dLDLF [42]	2.24	–	RS
DRFs [32]	2.17	91.3%	RS
BridgeNet [44]	2.38	–	RS
VGG16 + ODL($\lambda =0$)	2.30	91.1%	RS
VGG16 + ODL($\lambda =0.25$)(Ours)	2.16	92.9%	RS

Equations

$$D_{k}^{+}=\Big{\{}(\bm{x}_{i},1)\Big{|}y_{i}>r_{k}\Big{\}},D_{k}^{-}=\Big{\{}(\bm{x}_{i},0)\Big{|}y_{i}\leq r_{k}\Big{\}}.$$

$$h(\bm{x_{i}})=r_{1}+\eta\sum_{k=1}^{K-1}\mathbf{1}\Big{(}{f_{k}(\bm{x_{i}})}>0.5\Big{)},$$

$$\mathcal{L}_{c}=-\frac{1}{N}\sum_{i=1}^{N}\sum_{k=1}^{K-1}\Big{(}t_{i}^{k}\log(o_{i}^{k})+(1-t_{i}^{k})\log(1-o_{i}^{k})\Big{)},$$

$${\bar{p}}_i^k=\frac{\exp (o_i^k)}{{\sum }_{j=1}^{K-1}\exp (o_i^j)},p_i^k=\frac{\exp (t_i^k)}{{\sum }_{j=1}^{K-1}\exp (t_i^j)}.$$

$$\mathcal{L}_{\mathrm{EMD}}=\frac{1}{N}\sum_{i=1}^{N}\sum_{k=1}^{K-1}\Big{\{}CDF_{k}(\bm{\bar{p}}_{i})-CDF_{k}(\bm{p}_{i})\Big{\}}^{2},$$

$$\mathcal{L}={\mathcal{L}}_c+\lambda {\mathcal{L}}_{\mathrm{EMD}}$$

$$\frac{\partial \mathcal{L}}{\partial \mathbf{W}}=\frac{\partial {\mathcal{L}}_c}{\partial {\mathbf{o}}_i}\frac{\partial {\mathbf{o}}_i}{\partial \mathbf{W}}+\lambda \frac{\partial {\mathcal{L}}_{\mathrm{EMD}}}{\partial {\bar{\mathbf{p}}}_i}\frac{\partial {\bar{\mathbf{p}}}_i}{\partial \mathbf{W}},$$

$$\frac{\partial\mathcal{L}_{c}}{\partial o_{i}^{k}}=-\Big{(}\frac{t_{i}^{k}}{o_{i}^{k}}-\frac{1-t_{i}^{k}}{1-o_{i}^{k}}\Big{)}.$$

$$\frac{\partial {\mathcal{L}}_{\mathrm{EMD}}}{\partial {\bar{p}}_i^k}$$

$$=\frac{\partial {\sum }_{j=1}^{K-1}({\sum }_{\ell =1}^j({\bar{p}}_i^{\ell }-p_i^{\ell }){)}^2}{\partial {\bar{p}}_i^k}$$

$$=2\sum _{j=k}^{K-1}\sum _{\ell =1}^j({\bar{p}}_i^{\ell }-p_i^{\ell }),$$

$$\mathcal{L}_{gender}=-\frac{1}{N}\sum_{i=1}^{N}\Big{(}g_{i}\log(\hat{g_{i}})+(1-g_{i})\log(1-\hat{g_{i}})\Big{)},$$

Full text

1 Introduction

Human age estimation is an active research topic because age estimation plays an important role in many applications, such as video surveillance, social networking, and human–computer interaction. Existing age estimation methods are primarily based on facial images [2, 3, 22, 23, 31], which are very informative and easy to be estimated. However, the performance of face-based age estimation approaches will be compromised when the face is occluded, for example, by sunglasses or makeup. In addition, face-based age estimation becomes challenging if a person is far away from the camcorders, which often occurs in many video surveillance systems located at crossroads, airports, and railway stations. As a unique biometric feature that can be perceived efficiently at a distance, using a person’s gait can be an alternative way to predict age when a facial image is not sufficiently informative or not available. Gait-based estimation has a psychological foundation that cannot be easily faked [7]. For example, an old person might walk slowly and hobble, whereas a young person might walk briskly.

In the field of gait-based age estimation, gait energy image (GEI) [15, 21], which compresses one or more gait sequences into a single image (as shown in Fig. 1), is one of the most widely used gait templates due to its simplicity and effectiveness. Some studies have applied age manifold learning techniques on GEIs to learn a low-dimensional representation that captures the intrinsic data distribution and geometric structure [16, 17]. Existing gait-based age estimation approaches can be roughly categorized as classification-based  [15] and regression-based methods [17, 21]. However, neither of them considers the ordinal relationship between age labels, which is an important clue for age estimation. Therefore, to address this issue, ranking-based methods  [3, 5, 13, 22] have been proposed. Typically, these methods decompose the ordinal regression into a series of binary classifications and utilize the cross-entropy loss to optimize binary classifications. However, the cross-entropy loss treats these classifications independently, ignoring the inner relationships among them. For these ordered binary classifications, the expected inner relationship is that the predictive probability of the $k$-th classifier should not be greater than the probability of the ($k-1$)-th classifier, as explained in Fig. 2.

This paper proposes an ordinal distribution regression with a global and local convolutional neural network, which we refer to as ODR-GLCNN, for gait-based age estimation. Similar to ranking-based methods for facial-based age estimation, we consider gait-based age estimation as an ordinal regression and decompose the ordinal regression problem into a series of binary classification sub-problems. Note that the primary issue with existing ranking-based methods is that they solve these binary sub-problems independently. Consequently, such methods neglect the inner relationship to some extent and do not make good use of the correlation between these sub-binary tasks. To address this limitation, we propose an ordinal distribution loss (ODL) to penalize the distribution difference between the estimated and ground-truth age. Similar to Hou et al.’s work [10], this proposed loss also utilizes the squared Earth mover’s distance (EMD) to penalize the distribution discrepancy. Unlike Hou et al. [10], which utilizes the squared EMD loss on the classification task, we use the loss in an ordinal regression task, e.g., gait-based age estimation. To the best of our knowledge, this is the first time that the distribution discrepancy on ordinal regression through squared EMD loss has been considered. In addition, we propose a unique network, consisting of a global and three local sub-networks, to obtain a global structure and local structures from the head, body, and feet. Experimental results on the OULP-Age dataset [40] and the MORPH Album II dataset [26] demonstrate that the proposed approach outperforms state-of-the-art methods on both gait-based and face-based age estimation.

The contributions of this paper are: 1) A deep ordinal distribution regression for gait-based age estimation is proposed, which achieves state-of-the-art predictive performance on the OULP-Age dataset; 2) A novel network, named as GL-CNN, comprising a global network and three local sub-networks is proposed to learn more representative features from the gait globally and locally; 3) An ordinal distribution loss (ODL) is proposed to consider the inner relationships among a series of binary sub-problems.

2 Related work

In this section, we provide a brief survey of face- and gait-based age estimation studies as well as ordinal regression studies.

Face-based age estimation: Existing face-based age estimation approaches can be categorized as classification-, regression-, and ranking-based methods. Classification-based methods are often used to roughly estimate the age group of a subject in a face image [12, 41]. Different ages or age groups are treated as independent classes. However, these methods do not adequately consider the cost difference of subjects belonging to different age groups. Regression-based methods provide a more accurate age assessment from a facial image  [6, 39]. Typically, regression-based methods employ Euclidean loss ($\ell_{2}$ loss) to penalize the difference between the estimated age and the ground-truth age. Recently, ranking-based or ordinal methods have been proposed for facial age estimation [1, 2, 22, 3]. These approaches consider age as an ordered label and use multiple binary classifiers to determine the rank of a specific age. Unlike the $\ell_{2}$ loss, which ignores the ordinal information, ranking-based methods can explicitly model the ordinal relationships among face images sampled from different ages.

Gait-based age estimation: The earliest gait-based age estimation study was that of  [21], where a Gaussian process regression (GPR) [25] was introduced to predict age from human gait. Then, GPR was refined with an active set method [37] to reduce the computational time required for online age estimation [20]. Lu and Tan proposed a multi-label guided (MLG) subspace to better characterize the feature space by correlating the age and gender information of subjects [15]. They further proposed an ordinary preserving manifold learning approach to seek a low-dimensional discriminative subspace for age estimation [17]. Considering the age variations within different groups, such as children, adults, and the elderly, Li et al. proposed an age group-dependent manifold method [14]. In that method, after an age group classifier was trained, a kernel support vector machine (SVM) regression was added to provide accurate assessment in each age group. This method achieved state-of-the-art gait-based age estimation performance.

Ordinal regression: Most ordinal algorithms can be considered as refined versions of classification algorithms with ordinal constraints [4, 9, 30]. For example, Herbrich et al. utilized an SVM for ordinal regression [9]. Then, Shashua and Levin refined the SVM to handle multiple thresholds [30]. Crammer and Singer proposed the perceptron ranking algorithm to generalize an online perceptron algorithm with multiple thresholds for ordinal regression [4]. Another way to utilize classification algorithms directly is to transfer ordinal regression into a series of simpler binary classifications [5, 13]. Specifically, Frank and Hall utilized a decision tree for binary classifications for ordinal regression [5]. Li and Lin learned ordinal regression using a set of classifiers, and then employed an SVM for the final classification [13]. Recently, Niu et al. introduced a CNN with multiple binary outputs to solve the ordinal regression for age estimation [22]. Ordinal regression was also used in [3] by learning multiple binary CNNs and aggregating the final outputs. However, these ordinal regression methods solve each binary sub-problem independently and do not consider the underlying relationships among these binary sub-problems. Thus, this paper proposes a distribution loss to utilize such a relationship to improve the accuracy of age estimation.

3 Proposed method

In this section, we describe the proposed ordinal regression for gait-based age estimation, the network comprising a single global and three local sub-networks, and a unique distribution loss in more detail.

3.1 Ordinal regression

We treat gait-based age estimation as an ordinal regression in order to utilize the ordinal relationship of age labels. Let $\bm{x}_{i}\in\mathcal{X}$ denote the $i$-th input GEI sample. The corresponding age is $y_{i}\in\mathcal{Y}=\{r_{1},r_{2},\dots,r_{K}\}$ with ordered ranks $r_{K}\succeq r_{K-1}\succeq\dots\succeq r_{2}\succeq r_{1}$. The symbol $\succeq$ denotes the order among different ranks. Given a training set $\mathcal{D}=\{(\bm{x}_{i},y_{i})\}_{i=1}^{N}$, the ordinal regression is to learn a mapping from images to ranks, i.e., $h(\cdot):\mathcal{X}\to\mathcal{Y}$.

Inspired by two ranking-based methods [3, 22], we decompose ordinal regression into a series of binary classifications. Specifically, the ordinal regression with $K$ ranks is decomposed into $K-1$ binary classifiers $\{f_{k}\}_{k=1}^{K-1}$. For each $r_{k}\in\{r_{1},r_{2},\dots,r_{K-1}\}$, a binary classifier is constructed to predict whether the rank of a sample $y_{i}$ is greater than that of $r_{k}$. The final rank of an unknown test sample is determined by summarizing all results of the $K-1$ binary classifiers.

To train the $k$-th binary classifier $f_{k}$, the given dataset $\mathcal{D}$ is divided into two subsets (one positive class and one negative class) determined by whether the age is greater than $k$, i.e.,

[TABLE]

All $K-1$ binary classifiers are well-trained with their respective training datasets. The age of the test sample $\bm{x_{i}}$ is predicted as follows:

[TABLE]

where $f_{k}(\bm{x_{i}})\in[0,1]$ is the output probability of the $k$-th classifier for the sample $\bm{x_{i}}$ (i.e., the $k$-th output of GL-CNN); $\eta$ is the partitioning interval; and $\mathbf{1}(\cdot)$ denotes the truth-test operator, which is 1 if the inner condition holds and 0 otherwise.

3.2 GL-CNN

Fig. 3 presents an overview of the proposed GL-CNN for gait-based age estimation. The proposed network comprises a single global and three local CNNs, followed by three fully connected layers with $K-1$ outputs. Next, we describe the network in detail.

Grayscale GEIs of size $128\times 88$ are input to the global network. Considering that different parts of a gait take on different local behaviors, we crop the GEI template into three parts: head, body, and feet. In the OULP-Age dataset [40], the gait images of various people are detected, cropped, aligned, and resized to a uniform silhouette template of same height. In this study, the three parts are cropped using the following three boxes without overlap: $22\times 88$, $48\times 88$, and $58\times 88$. Then, three local networks are designed to learn finer details from these three parts separately. More specifically, there are three convolutional layers in both global and local sub-networks. At the first convolutional layer, 32 filters of size $7\times 7$ with a stride of 1 are applied to the input images, followed by a Leaky Rectified Linear Unit (LeakyReLU) [18]. Then, a max-pooling operation with filters of size $2\times 2$ applied with a stride of 2 is used to emphasize the strongest responsive points in the feature maps. Similar operations are performed at the second and third convolutional layers with different filter sizes (Fig. 3). Note that we concatenate the three local feature maps from the second convolution layers along the height dimension to form new local feature maps in a local network. The local network is further concatenated with the feature maps from the global network along the channel dimension.

Then, there are three fully connected layers, as shown in Fig. 3. Among them, F4 is the first fully connected layer in which the feature maps are flattened into a feature vector. There are 1024 neurons in F4, followed by a LeakyReLU and a dropout layer [35]. F5 is the second fully connected layer with 1024 neurons. The second fully connected layer receives the output from F4, followed by a LeakyReLU and another dropout layer. F6 is the third fully connected layer with $K-1$ neurons. This layer receives the output from F5, followed by a LeakyReLU and a dropout layer. Through a sigmoid layer, the $K-1$ output corresponds to the predictive probabilities from $K-1$ binary classifiers. Typically, the network parameters are optimized by minimizing the objective function.

3.3 ODL

Here we cast the age label $y_{i}$ as $\bm{t}_{i}=[t_{i}^{1},t_{i}^{2},\dots,t_{i}^{K-1}]^{T}$ for $K-1$ binary classifiers, where $t_{i}^{k}=\mathbf{1}(y_{i}>r_{k})$. We employ cross-entropy loss as the loss function for these binary classifiers. The loss can be calculated as

[TABLE]

where $o_{i}^{k}$ is the output value of the $k$-th binary classifier for the $i$-th sample. However, the cross-entropy loss optimizes these binary classifiers separately, resulting in a discrepancy between different binary classifications, as shown in Fig. 2.

To fully utilize the inner relationships among these $K-1$ outputs, we consider these outputs as a probability distribution and then propose a distribution loss, e.g., $EMD^{2}$ [10], to penalize the discrepancy between the output distribution and ground-truth distribution. First, the output values are transformed into the probability value by the softmax function,

[TABLE]

Then, the $EMD^{2}$ loss is defined as

[TABLE]

where $\bm{\bar{p}_{i}}=[\bar{p}_{i}^{1},\bar{p}_{i}^{2},\cdots,\bar{p}_{i}^{K-1}]^{T}$ and $\bm{p}_{i}=[p_{i}^{1},p_{i}^{2},\cdots,p_{i}^{K-1}]^{T}$ are the probability distributions corresponding to the $i$-th output $\bm{o}_{i}$ and the $i$-th ground truth $\bm{t}_{i}$, respectively. $CDF(\cdot)$ is a cumulative density function of its input, and $CDF_{k}(\cdot)$ is the $k$-th element of the CDF of its input.

Finally, we propose an ODL by combining the cross-entropy loss with the $EMD^{2}$ loss. This loss function is easily embedded into the GL-CNN architecture for end-to-end learning. The ODL is

[TABLE]

where $\lambda$ is a hyper-parameter that controls the influence of $\mathcal{L}_{\mathrm{EMD}}$ in the joint loss.

3.4 Learning ODR-GLCNN

One advantage of using Eq. (6) is that the ODL can simultaneously learn each binary classification and the inner relationship between these binary classifications. For the $i$-th sample $\bm{x}_{i}$, the gradient of our loss can be derived as

[TABLE]

where $\bm{W}$ represents the network parameters, and $\frac{\partial\bm{o}_{i}}{\partial\bm{W}}$ and $\frac{\partial\bm{\bar{p}}_{i}}{\partial\bm{W}}$ can be derived through the standard backpropagation method. For the $k$-th element of $\bm{o}_{i}$, the gradient can be derived as

[TABLE]

For the $k$-th element of $\bm{\bar{p}_{i}}$, the gradient can be derived as

[TABLE]

where $k\in\{1,2,\dots,K-1\}$. Eq. (8) indicates that the gradient of the cross-entropy loss is only related to the output value of each binary classification and its corresponding ground truth, ignoring the intrinsic correlation for their binary classifiers. In contrast, the output value of each classification is considered when computing the gradient of a specific binary classification in $EMD^{2}$ loss, as shown in Eq. (9). Therefore, the ODL considers each binary classification and can utilize the inner relationships among them.

4 Experiments

In this section, we describe the experimental setting and demonstrate the effectiveness of the proposed method by comparing it with state-of-the-art methods and performing a set of ablative studies on the OULP-Age gait dataset [40]. In addition, we evaluate the generalization ability of the proposed approach to other tasks, e.g., facial age estimation on the MORPH Album II dataset [26].

4.1 Experimental setting

4.1.1 Data preparation

OULP-Age is the largest gait dataset in the world to date, containing 63,846 GEIs (31,093 males and 32,753 females; age 2 to 90 years; sample size $128\times 88$). Gender distribution by age in five-year intervals is shown in Fig. 4. As Xu et at. [40] suggests, the OULP-Age dataset is averagely divided into two disjoint subsets (training set and test set). The training set contains 15,596 males and 16,327 females, and the test set contains 15,497 males and 16,426 females. Note that both datasets have similar age distribution.

Note that USF [29] is another popular gait dataset for age estimation. However, it is too small (only 122 subjects) and rough (age labels are not very accurate) to be used for evaluating the performance of deep-learning-based gait age estimation. Therefore, we trained the proposed method on the OULP-Age dataset and used the USF dataset only for testing.

MORPH Album II is one of the largest and most popular longitudinal face databases for age estimation in the public domain (55,134 face images, 13,617 subjects, age 16 to 77 years)  [26]. To further demonstrate the effectiveness of the ODL, we evaluated the proposed loss on the MORPH Album II dataset. Following the previous studies  [22, 3, 23, 32], we used the five-fold random split (RS) protocol to evaluate the facial age estimation performance. All face images were aligned based on five facial landmarks detected using the open-source SeetaFaceEngine face alignment algorithm111https://github.com/seetaface/SeetaFaceEngine and then were resized to $256\times 256\times 3$.

4.1.2 Evaluation metrics

The age estimation performance was evaluated using the mean absolute error (MAE) and the cumulative score (CS). MAE represents the average of the absolute errors between the predicted age and the ground truth over all test samples. MAE is defined as \frac{1}{N}\sum_{i=1}^{N}\Big{|}h(\bm{x}_{i})-y_{i}\Big{|}, where $N$ is the total number of test samples. CS is calculated as $CS(k)=\frac{N_{k}}{N}\times 100\%$, where $N_{k}$ is the number of test samples whose absolute error between the estimated age and the ground truth is not greater than $k$ years. CS reveals a consistent performance by computing the accuracy of the evaluated model at different $k$-levels.

4.2 Gait-based age estimation results

4.2.1 Implementation details

In our experiments, we utilized our GL-CNN, a CNN (comprising a global part, as shown in Fig. 3), and a VGG16 [24, 33] as three backbone networks. We used Adam [11] with a learning rate of $0.0001$, beta1 0.5, beta2 0.999, weight decay of $0.00001$, batch size of 300, and maximal epochs of 300 for CNN and GL-CNN. Following a previous study  [23], we used stochastic gradient descent with a learning rate of $0.0001$, weight decay of $0.0001$, batch size of 300, and maximal epochs of 100 for VGG16. The learning rate was reduced by multiplying the effective rate by 0.1 every 15 epochs. To make the grayscale GEIs suitable for VGG16, we copied the GEIs three times as RGB channels to feed into the VGG16, which was pre-trained on ImageNet [28]. In addition, the weight coefficient of the $EMD^{2}$ loss term in Eq. (6) was set to $\lambda=10$, which was tuned according to the model performance. All experiments were implemented on PyTorch with four GeForce GTX 1080 Ti GPUs.

4.2.2 Comparison of the proposed method to state-of-the-art methods on the OULP-Age dataset

We compared the proposed method with state-of-the-art methods, including classification-based methods (e.g., MLG [15]), regression-based methods (e.g., GPR [21], SVR [34], and ASSOLPP [14]), and age manifold learning-based methods (e.g., OPLDA and OPMFA [16]). In addition, we implemented a deep learning method as a baseline, i.e., VGG16 + Mean-Variance, to validate the effectiveness of the proposed method. This method  [23] achieves an outstanding performance in the field of face-based age estimation.

Table 1 shows the result obtained by eight methods on the OULP-Age dataset. This suggests that CNN-based methods, such as [23], perform better than traditional methods in terms of MAE [14, 15, 16, 21, 34]. CNN-based methods demonstrate better performance because they have many more parameters and learn more representative features with end-to-end training. The proposed method performs the best among all approaches because it benefits from both a more representative feature extraction network (GL-CNN) and a unique loss function (the ODL), which can learn each binary classification of ordinal regression and the inner relationships among them. In addition, as shown in Fig. 5, the CS results on the OULP-Age dataset further demonstrate that the proposed approach performs consistently better than the other state-of-the-art methods.

Some age estimation examples are shown in Fig. 6. As seen, the proposed approach is quite robust for young, middle-aged, and old subjects. However, as is noticeable from the last row of Fig. 6, age estimation accuracy may degenerate when a person wears heavy clothes or when a person is extremely thin or overweight.

4.2.3 Testing with state-of-the-art methods on the USF dataset

To further demonstrate the effectiveness of the proposed method, we evaluated the performance of our model on the USF dataset and compared it with state-of-the-art methods. As described above, the USF dataset is too small to train a deep network model. Therefore, we trained our model and other methods on the OULP-Age dataset, and then tested using the USF dataset.

Table 2 shows that for the USF gallery (probe A) subset, the proposed method outperforms SVR, MLG, OPLDA, OPMFA, ASSOLPP, and VGG16+Mean-Variance with MAE gains of 2.3 (2.08), 3.54 (3.31), 1.14 (1.01), 1.04 (0.87), 0.9 (0.73), and 0.19 (0.18) years old, respectively. The results further indicate that the proposed method consistently outperforms other methods for gait-based age estimation.

4.2.4 Analyzing the performance of GL-CNN

We evaluated the performance of the proposed GL-CNN by comparing it with a simple CNN comprising a global part and the VGG16 network, which is widely used for age estimation based on gait. All three networks used cross-entropy loss as the loss function. The MAE and CS ($k=5$) results for these three different networks are shown in Table 3.

Compared to a simple CNN, GL-CNN achieves better age estimation performance. Table 3 shows that 1) compared to CNN and VGG16, GL-CNN achieves the best performance in two criteria; 2) although VGG16 has more parameters, GL-CNN effectively learns more detailed information by combining a global and three local structures, resulting in an improved performance; and 3) the computational cost of GL-CNN is only slightly higher than that of CNN and much smaller than that of VGG16.

To better demonstrate the effectiveness of the proposed network, we visualize CNN and GL-CNN features through $t$-distributed stochastic neighbor embedding [19] ($t$-SNE) technique with perplexity 30, as shown in Fig. 7. For better visualization, the age label is divided into 9 age groups: e.g. $[0\sim 10],[11\sim 20],[21\sim 30],\cdots,[81\sim 90]$. As can be seen, both GL-CNN and CNN features appear to maintain a manifold-like structure because the order of ages varies smoothly from left to right. However, after zooming in Fig. 7, it is evident that the inner age group samples of GL-CNN are denser than those of CNN, particularly in age groups $[21\sim 30]$ and $[31\sim 40]$, which shows that GL-CNN can achieve fewer age estimation errors because it learns a better feature representation from both global and local structures.

4.2.5 Comparison of different losses

To validate the effectiveness of the proposed ODL, we compared it with three widely used losses in gait-based age estimation tasks, e.g., Euclidean, MAE, and cross-entropy losses, by performing age estimation based on the proposed GL-CNN. The MAE and CS ($k=5$) of these losses are reported in Table 4.

It can be seen that cross-entropy loss outperforms Euclidean loss and MAE loss for the age estimation tasks. Euclidean and MAE losses easily lead to over-fitting and do not consider the ordinal information between age labels. In contrast, the proposed ODL incorporates the inner relationship between the binary classifications by using a distribution loss, i.e., $EMD^{2}$ loss, resulting in better predictive performance.

In addition, to demonstrate the difference between our proposed ODL and the squared EMD loss in  [10], we compare them on the OULP-Age dataset. Following [10], the squared EMD loss is utilized on a classification task in these experiments. The comparison results, shown in Table 4, indicate that the combination of squared EMD loss and ordinal regression is more suitable for the gait-based age estimation task. The results indicate that utilizing squared EMD loss with ordinal regression is reasonable and effective.

4.2.6 Influence of gender

We realize that gender is correlated with age because human gait varies between males and females even within the same age group, as shown in Fig. 1. To better utilize the relationship between age and gender, we embedded a multi-task technique into the CNN-based framework. As shown in Fig. 8, specifically, we investigated a gender classification task for the proposed method and three other CNN-based methods. As a binary classification, the gender loss is defined as

[TABLE]

where $g_{i}$ is the ground truth of gender for the $i$-th sample and $\hat{g_{i}}$ is the corresponding predicted value.

Table 5 indicates that gender information indeed improves the performance of gait-based age estimation. Moreover, the accuracy of gender classification with the proposed method is 97.8%, implying that, as a byproduct, the proposed network can accurately predict gender from the gait.

4.3 Facial age estimation

We applied the proposed ODL to facial age estimation using the MORPH Album II dataset, and compared the results with those of state-of-the-art methods [22, 27, 3, 42, 23, 32, 43, 44]. Following previous studies,  [23, 32], we also utilized VGG16, pre-trained with ImageNet [28], as the backbone network with the proposed ODL. The results of individual approaches in terms of MAE and CS are reported in Table 6. As can be seen, our approach achieves better prediction performance than the state-of-the-art method DRF [32], which suggests that the proposed approach can be well-generalized to the facial age estimation task. In addition, the results obtained using ODL ($\lambda=0.25$) are better than those obtained using a single cross-entropy loss ($\lambda=0$ when calculating ODL), which indicates that ODL is more effective in learning the ordinal relationships among different ages than a single cross-entropy loss.

5 Conclusion

In this paper, we proposed an ordinal distribution regression, including a novel network GL-CNN and a useful loss function ODL, for gait-based age estimation. Specifically, the GL-CNN consisting of one global and three local sub-networks was constructed to extract more representative gait features. And the proposed ODL incorporating cross-entropy loss and $EMD^{2}$ loss was found to be more effective in learning the ordinal relationships among different ages than a single cross-entropy loss. We also found that if the gender information is available for training, embedding a multi-task strategy into the proposed framework can more or less improve the age estimation performance. Experiments on the OULP-Age and MORPH Album II datasets show that the proposed approach outperforms state-of-the-art methods on gait-based age estimation and generalize well for facial age estimation tasks.

In the future, it is worth studying how to utilize temporal information or cross-view information [8, 36, 38] of gait sequences to improve the accuracy and effectiveness of gait-based age estimation.

Acknowledgements

This work supported in part by the National Natural Science Foundation of China (NSFC61673118), Shanghai Municipal Science and Technology Major Project (No.2018SHZDZX01), ZJLab, and Shanghai Pujiang Program (No.16PJD009). We are grateful to the reviewers and the Associate Editor for their constructive comments.