A Strong Baseline and Batch Normalization Neck for Deep Person Re-identification

TL;DR

This paper presents a simple yet effective baseline for person re-identification using deep neural networks, introducing a novel batch normalization neck (BNNeck) that improves performance by separating metric and classification features.

Contribution

It introduces BNNeck, a new structure that enhances person ReID performance, and evaluates effective training tricks to establish a strong baseline.

Findings

Achieves 94.5% rank-1 accuracy on Market1501 with ResNet50.

BNNeck significantly boosts baseline performance.

Baseline surpasses existing global- and part-based methods.

Abstract

This study explores a simple but strong baseline for person re-identification (ReID). Person ReID with deep neural networks has progressed and achieved high performance in recent years. However, many state-of-the-art methods design complex network structures and concatenate multi-branch features. In the literature, some effective training tricks briefly appear in several papers or source codes. The present study collects and evaluates these effective training tricks in person ReID. By combining these tricks, the model achieves 94.5% rank-1 and 85.9% mean average precision on Market1501 with only using the global features of ResNet50. The performance surpasses all existing global- and part-based baselines in person ReID. We propose a novel neck structure named as batch normalization neck (BNNeck). BNNeck adds a batch normalization layer after global pooling layer to separate metric and classification losses into two different feature spaces because we observe they are inconsistent in one embedding space. Extended experiments show that BNNeck can boost the baseline, and our baseline can improve the performance of existing state-of-the-art methods. Our codes and models are available at: https://github.com/michuanhaohao/reid-strong-baseline.

Tables

Table 1. TABLE I: Performance of different models is evaluated on Market1501 and DukeMTMC-reID datasets. Baseline-S represents the standard baseline.

	Market1501		DukeMTMC
Model	r = 1	mAP	r = 1	mAP
Baseline-S	87.7	74.0	79.7	63.7
+warmup	88.7	75.2	80.6	65.1
+REA	91.3	79.3	81.5	68.3
+LS	91.4	80.3	82.4	69.3
+stride=1	92.0	81.7	82.6	70.6
+BNNeck	94.1	85.7	86.2	75.9
+center loss	94.5	85.9	86.4	76.4

Table 2. TABLE II: The performance of different models is evaluated on cross-domain datasets. M → → \to D means that we train the model on Market1501 and evaluate it on DukeMTMC-reID.

	M$\to$D		D$\to$M
Model	r = 1	mAP	r = 1	mAP
Baseline	24.4	12.9	34.2	14.5
+warmup	26.3	14.1	39.7	17.4
+REA	21.5	10.2	32.5	13.5
+LS	23.2	11.3	36.5	14.9
+stride=1	23.1	11.8	37.1	15.4
+BNNeck	26.7	15.2	47.7	21.6
+center loss	27.5	15.0	47.4	21.4
-REA	41.4	25.7	54.3	25.5

Table 3. TABLE III: Ablation study of different neck structures in Fig. 8 .

		Market1501		DukeMTMC
Feature	Metric	r = 1	mAP	r = 1	mAP
$f$	Neck1	89.4	77.5	78.9	65.3
$f$	Neck2	91.0	80.9	82.5	69.4
$f$	Neck3	92.0	81.7	82.6	70.6
$f_i$	BNNeck1	93.1	83.9	85.2	74.0
$f_i$	BNNeck2	90.3	79.1	82.5	67.9
$f_i$	BNNeck3	92.5	81.8	83.3	71.9
$f_t$	BNNeck	94.2	85.5	85.7	74.4
$f_i$	BNNeck	94.1	85.7	86.2	75.9

Table 4. TABLE IV: Ablation study of BNNeck. f 𝑓 f (w/o BNNeck) is baseline without BNNeck. BNNeck includes features f t subscript 𝑓 𝑡 f_{t} and f i subscript 𝑓 𝑖 f_{i} . We evaluate their performance with Euclidean and cosine distance.

		Market1501		DukeMTMC
Feature	Metric	r = 1	mAP	r = 1	mAP
$f$ (w/o BNNeck)	Euclidean	92.0	81.7	82.6	70.6
$f_t$	Euclidean	94.2	85.5	85.7	74.4
$f_t$	Cosine	94.2	85.7	85.5	74.6
$f_i$	Euclidean	93.8	83.7	86.6	73.0
$f_i$	Cosine	94.1	85.7	86.2	75.9

Table 5. TABLE V: Evaluation with different weights of center loss β 𝛽 \beta . R 𝑅 R is the ratio of intra-class distance to inter-class distance.

		Market1501			DukeMTMC
Feature	$\beta$	r = 1	mAP	$R$	r = 1	mAP	$R$
	0	94.2	85.5	0.407	85.7	74.4	0.424
	0.0005	93.9	85.7	0.405	86.5	75.1	0.420
$f_t$	0.005	94.2	85.7	0.394	86.2	75.4	0.417
	0.05	94.4	85.4	0.365	86.4	74.9	0.403
	0.5	92.6	81.1	0.311	85.5	72.2	0.363
	0	94.1	85.7	0.590	86.2	75.9	0.568
	0.0005	94.5	85.9	0.589	86.4	76.4	0.564
$f_i$	0.005	94.3	85.9	0.595	86.8	76.4	0.566
	0.05	94.3	85.7	0.592	86.7	76.5	0.560
	0.5	94.1	84.7	0.593	87.4	76.9	0.554

Table 6. TABLE VI: Comparison of different baselines on Market1501 and DukeMTMC-reID datasets. ID and Tri stands for ID loss and triplet-based loss, respectively.

		Market1501		DukeMTMC
Baseline	Loss	r = 1	mAP	r = 1	mAP
IDE[8]	ID	79.5	59.9	-	-
TriNet[49]	Tri	84.9	69.1	-	-
AWTL[50]	Tri	89.5	75.7	79.8	63.4
GP[9]	ID	91.7	78.8	83.4	68.8
PCB[5]	ID	92.3	77.4	81.7	66.9
Our	ID+Tri	94.5	85.9	86.4	76.4

Table 7. TABLE VII: Comparison of state-or-the-art methods. N f subscript 𝑁 𝑓 N_{f} is the number of features used in the inference stage.

			Market1501		DukeMTMC
Type	Method	$N_f$	r = 1	mAP	r = 1	mAP
Pose-guided	GLAD[19]	4	89.9	73.9	-	-
	PIE [21]	3	87.7	69.0	79.8	62.0
	PSE [20]	3	78.7	56.0	-	-
Mask-guided	SPReID [23]	5	92.5	81.3	84.4	71.0
	MaskReID [24]	3	90.0	75.3	78.8	61.9
Stripe-based	AlignedReID++ [3]	1	90.6	77.7	81.2	67.4
	SCPNet [17]	1	91.2	75.2	80.3	62.6
	PCB+RPP [5]	6	93.8	81.6	83.3	69.2
	Pyramid[51]	1	92.8	82.1	-	-
	Pyramid[51]	21	95.7	88.2	89.0	79.0
	BFE[52]	2	94.5	85.0	88.7	75.8
	MGN [16]	1	89.8	78.5	-	-
	MGN [16]	8	95.7	86.9	88.7	78.4
Attention-based	Mancs [4]	1	93.1	82.3	84.9	71.8
	DuATM [25]	1	91.4	76.6	81.2	62.3
	HA-CNN [26]	4	91.2	75.7	80.5	63.8
GAN-based	Camstyle [31]	1	88.1	68.7	75.3	53.5
	PN-GAN [32]	9	89.4	72.6	73.6	53.2
Global feature	IDE [8]	1	79.5	59.9	-	-
	SVDNet [53]	1	82.3	62.1	76.7	56.8
	TriNet[49]	1	84.9	69.1	-	-
	AWTL[50]	1	89.5	75.7	79.8	63.4
	Ours	1	94.5	85.9	86.4	76.4

Table 8. TABLE VIII: Performance of some state-of-the-art methods reproduced by our strong baseline. The values in parentheses are the results reported by authors.

		Market1501		DukeMTMC
Method	Reference	r = 1	mAP	r = 1	mAP	Loss
Baseline1	BNNeck1	93.1	83.9	85.2	74.0	$L_{ID}$
Baseline2	BNNeck	94.1	85.7	86.2	75.9	$L_{ID}$, $L_{Tri}$
Baseline3	BNNeck	94.5	85.9	86.4	76.4	$L_{ID}$, $L_{Tri}$, $L_C$
$k$-reciprocal[9]	CVPR17	95.4(77.1)	94.2(63.6)	90.3(-)	89.1(-)	$L_{ID}$, $L_{Tri}$, $L_C$
PCB[5]	ECCV18	94.0(92.3)	84.0(77.4)	88.6(81.7)	77.2(66.1)	$L_{ID}$, $L_{Tri}$
AligedReID++ [3]	PR19	94.3(91.8)	86.5(79.1)	86.5(82.1)	76.9(69.7)	$L_{ID}$, $L_{Tri}$
CamStyle[31]	TIP19	93.3(88.1)	81.0(68.7)	80.3(75.3)	60.1(53.5)	$L_{ID}$
MGN[16]	ACMMM19	95.3(95.7)	86.3(86.9)	89.2(88.7)	78.9(78.4)	$L_{ID}$, $L_{Tri}$

Table 9. TABLE IX: Performance of our baseline with different backbone.

	Market1501		DukeMTMC
Backbone	r = 1	mAP	r = 1	mAP
ResNet18	91.7	77.8	82.5	68.8
ResNet34	92.7	82.7	86.4	73.6
ResNet50	94.5	85.9	86.4	76.4
ResNet101	94.5	87.1	87.6	77.6
SeResNet50	94.4	86.3	86.4	76.5
SeResNet101	94.6	87.3	87.5	78.0
SeResNeXt50	94.9	87.6	88.0	78.3
SeResNeXt101	95.0	88.0	88.4	79.0
IBN-Net50-a	95.0	88.2	90.1	79.1
IBN-Net50-b	93.5	83.9	86.4	73.5

Equations

$$lr(t)=\left\{\begin{array}[]{ll}{3.5\times 10^{-4}\times\frac{t}{10}}&{\textnormal{ if }t\leq 10}\\ {3.5\times 10^{-4}}&{\textnormal{ if }10<t\leq 40}\\ {3.5\times 10^{-5}}&{\textnormal{ if }40<t\leq 70}\\ {3.5\times 10^{-6}}&{\textnormal{ if }70<t\leq 120}\\ \end{array}\right.$$

$$L(ID)=\sum_{i=1}^{N}-q_{i}\log\left(p_{i}\right)\left\{\begin{array}[]{ll}q_{i}=0,y\neq i\\ q_{i}=1,y=i\end{array}\right.$$

$$q_{i}=\left\{\begin{array}[]{ll}{1-\frac{N-1}{N}\varepsilon}&{\textnormal{ if }i=y}\\ {\varepsilon/N}&{\textnormal{ otherwise, }}\end{array}\right.$$

$$L_{Tri}=[d_p-d_n+\alpha {]}_{+},$$

$${\mathcal{L}}_C=\frac{1}{2}\sum _{j=1}^B{\Vert {\mathbf{f}}_{t_j}-{\mathbf{c}}_{y_j}\Vert }_2^2,$$

$$L=L_{ID}+L_{Triplet}+\beta L_C$$

Full text

I Introduction

Person re-identification (ReID) is widely applied in video surveillance and criminal investigation applications [2]. Person ReID with deep neural networks has progressed and achieved high performance in recent years [3, 4, 5]. Apart from many novel and effective ideas being proposed, the improvement of baseline model plays a key role. For example, Jian et al. [6, 7] proposed a baseline that greatly promoted development of underwater saliency detection. The importance of baseline model should not be ignored. However, few works [8, 9, 5] have focused on the design of an effective baseline. The performance of such baselines has gradually become obsolete due to the rapid development of person ReID. In the literature, some effective training tricks or refinements briefly appear in several papers or source codes. In the present study, we design a strong and effective baseline for person ReID by collecting and evaluating such effective training tricks.

This study has three motivations. First, we survey papers published on ECCV and CVPR in 2018. As shown in Fig. 1, many previous works were expanded on poor baselines. Only two in 23 baselines surpassed 90% rank-1 accuracy on Market1501. The rank-1 accuracies of four baselines were even lower than 80%. Achieving improvements on poor baselines cannot strictly demonstrate the effectiveness of some methods. Thus, a strong baseline is crucial in promoting research development.

Second, we discover that the improvements of some works were mainly from training tricks rather than methods themselves. So they were unfairly compared with other state-of-the-art methods. However, the training tricks were understated in the paper; thus, readers ignored them, thereby exaggerating the effectiveness of the method. We suggest that reviewers consider these tricks when commenting on academic papers.

Third, the industry prefers simple but effective models over concatenating many local features in the inference stage. In order to achieve great performance, researchers in the academia always combine several local features or use semantic information from pose estimation or segmentation models. Nevertheless, such methods bring extra consumption. Large features also greatly reduce the speed of the retrieval process. Thus, we use tricks to improve the capability of the ReID model and only use global features extracted by the model.

On the basis of the aforementioned considerations, the motivations of designing a strong baseline are summarized as follows:

• •

For the academia, we survey many works published on top conferences and discover that most of them were expanded on poor baselines. We aim to provide a strong baseline for researchers to achieve high accuracies in person ReID.

• •

For the community, we aim to provide references to reviewers regarding tricks that will affect the performance of the ReID model. We suggest that reviewers consider these tricks when comparing the performance of different methods.

• •

For the industry, we aim to provide effective tricks for acquiring improved models without extra consumption.

Many effective training tricks have been presented in papers or open-sourced projects. We collect tricks and evaluate each of them on ReID datasets. After numerous experiments, we select six tricks to introduce in this study. We propose a novel bottleneck structure, namely, batch normalization neck (BNNeck). As classification and metric losses are inconsistent in the same embedding space, BNNeck optimizes these two losses in two different embedding spaces. In addition, person ReID task mainly focuses on ranking performance, such as cumulative match characteristic (CMC) curve and mAP, but ignores the clustering effect, such as intra-class compactness and inter-class separability. However, clustering effect is important to some special tasks, such as object tracking, which must decide on a distance threshold to separate positive samples from negative ones. An easy approach to overcome this problem is to train the model with center loss. Finally, we add the tricks into a widely used baseline to obtain our modified baseline (the backbone is ResNet50), which achieves 94.5% and 85.9% mAP on Market1501.

To determine whether these tricks are generally useful or not, we design extended experiments from three aspects. First, we follow the cross-domain ReID settings in which the models are trained and evaluated on different datasets. Cross-domain experiments can show whether the tricks boost the models or simply suppress overfitting in the training dataset. Second, we evaluate all tricks with different backbones, such as ResNet18, SeResNet50, and IBNNet-50. All backbones achieve improvements from our training tricks. Third, we reproduce some state-of-the-art methods on our modified baseline. Experimental results show that our baseline obtains better performance than those reported in published papers. Although our baseline achieves surprising performance, some methods remain effective on our baseline. Thus, our baseline can be a strong baseline for the ReID community.

There are four main differences between the original version and this paper. 1)We add a new section Related Works to introduce the developments of deep person ReID. 2) We perform a lot of extended experiments to explain the effectiveness of BNNeck. 3) We explain why we introduce center loss and perform some extended experiments to analyze the effect of center loss. 4) We evaluate different backbones and some state-of-the-art methods on our baseline.

The contributions of this study are summarized as follows:

• •

We collect effective training tricks for person ReID. The improvements from each trick are evaluated on two widely used datasets.

• •

We observe the inconsistency between ID loss and triplet and propose a novel neck structure, namely, BNNeck.

• •

We observe that the ReID task ignores intra-class compactness and inter-class separability and claim that center loss can compensate for it.

• •

We proposed a strong ReID baseline. With ResNet50 backbone, it achieves 94.5% and 85.9% mAP on Market1501. To our best knowledge, this result is the best performance acquired by global features in person ReID.

• •

We design extended experiments to demonstrate that our baseline can be a strong baseline for the ReID community.

II Related Works

This section focuses on deep learning baseline for person ReID. In addition, existing approaches compared with our strong baseline for deep person ReID are introduced.

II-A Baseline for Deep Person ReID

Recent studies on person ReID mostly focus on building deep convolutional neural networks (CNNs) to represent the features of person images in an end-to-end learning manner. GoogleNet [10], ResNet [11], DenseNet [12], etc are widely used backbone networks. The baselines can be classified into two main genres in accordance with the loss function, i.e. classification loss and metric loss. For classification loss, Zheng et al. [8] proposed ID-discriminative embedding (IDE) to train the re-ID model as image classification which is fine-tuned from the ImageNet [13] pre-trained models. Classification loss is also called ID loss in person ReID because IDE is trained by classification loss. However, ID loss requires an extra fully connected (FC) layer to predict the logits of person IDs in the training stage. In the inference stage, such FC layer is removed, and the feature from the last pooling layer is used as the representation vector of the person image.

Different from ID loss, metric loss regards the ReID task as a clustering or ranking problem. The most widely used baseline based on metric learning is training model with triplet loss [14]. A triplet includes there images, i.e. anchor, positive, and negative samples. The anchor and positive samples belong to the same person ID, whereas the negative sample belongs to a different person ID. Triplet loss minimizes the distance from the anchor sample to the positive sample and maximizes the distance from the anchor sample to the negative one. However, triplet loss is greatly influenced by the sample triplets. Inspired by FaceNet[15], Hermans et al. proposed an online hard example mining for triplet loss (TriHard loss). Most current methods are expanded on the TriHard baseline. Combining ID loss with TriHard loss is also a popular manner of acquiring a strong baseline [3].

Apart from designing different losses, some works focus on building effective baseline model for deep person ReID. In [9], three good practices were proposed to build an effective CNN baseline toward person ReID. Their most important practice is adding a batch normalization (BN) layer after the global pooling layer. Similar to these models, the baseline uses a global feature for image representation. Sun et al. [5] proposed part-based convolutional baseline (PCB). Given an image input, PCB outputs a convolutional descriptor consisting of several part-level features. Both baselines have achieved good performance in person ReID.

II-B Some Existing Approaches for Deep person ReID

On the basis of the aforementioned baselines, many methods have been proposed in the past few years. We divide these works into striped-based, pose-guided, mask-guided, attention-based, GAN-based, and re-ranking methods.

Stripe-based methods, which divide the image into several stripes and extract local features for each stripe, play an important role in person ReID. Inspired by PCB, the typical methods includes AlignedReID++[3], MGN [16], SCPNet [17], etc. Stripe-based local features are effective in boosting the performance of the ReID model. However, they always encounter the problem of pose misalignment.

Pose-guided methods [18, 19, 20, 21] use an extra pose/skeleton estimation model to acquire human pose information. Pose information can exactly align corresponding parts of two person images. However, an extra model brings additional computation consumption. A trade off between the performance and speed of the model is important.

Mask-guided models [22, 23, 24] use mask as external cues to remove the background clutters in pixel level and contain body shape information. For example, Song et al. [22] proposed a mask-guided contrastive attention model that applies binary segmentation masks to learn features separately from the body and background regions. Kalayeh et al. [23] proposed SPReID, which uses human semantic parsing to harness local visual cues for person ReID. Mask-guided models extremely rely on accurate pedestrian segmentation model.

Attention-based methods [25, 26, 27, 28] involve an attention mechanism to extract additional discriminative features. In comparison with pixel-level masks, attention region can be regraded as an automatically learned high-level ‘mask’. A popular model is Harmonious Attention CNN (HA-CNN) model proposed by Li et al. [26]. HA-CNN combines the learning of soft pixel and hard regional attentions along with simultaneous optimization of feature representations. An advantage of attention-based models is that they do not require a segmentation model to acquire mask information.

GAN-based methods [29, 30, 31, 32, 33, 34] address the limited data for person ReID. Zheng et al. [29] first used GAN [35] to generate images for enriching ReID datasets. The GAN model randomly generates unlabeled and unclear images. On the basis of [29], PTGAN [30] and CamStyle [31] were proposed to bridge domain and camera gaps for person ReID, respectively. Qian et al. [32] proposed PNGAN for obtaining a new pedestrian feature and transforming a person into normalized poses. The final feature is obtained by combining the pose-independent features with original ReID features. With the development of GAN, many ganbased methods have been proposed to generate high quality for supervised and unsupervised person ReID tasks. Zhao et al. [34] integrated GAN with a widely used triplet loss method.

Re-ranking methods [36, 37, 38, 39] are post-processing strategies for image retrieval. In general, person ReID simply uses Euclidean or cosine distances in the retrieval stage. Zhong et al. [36] a $k$-reciprocal encoding method with the Jaccard distance of probe and gallery images to re-rank the ReID results. Shen et al. [37] proposed a deep group-shuffling random walk (DGRW) network for fully utilizing the affinity information between gallery images in training and testing processes. In the retrieval stage, DGRW can be regarded as a re-ranking method. Re-ranking is a critical step in improving retrieval accuracy.

III Our Strong Baseline and Training Tricks

A widely used baseline for the academia and industry is present in Fig. 2a. For convenience, such baseline is called standard baseline. Since we have introduced it in [1], the details of such standard baseline can be found in [1] and our open source code.

In addition, this section introduces some effective training tricks in person ReID. Our proposed BNNeck structure is discussed in detail. The intra-class compactness and inter-class separability problem for person ReID is also raised. Most tricks can be expanded on the standard baseline without changing the model architecture. Compared with our previous papaer [1], this paper does not add any new contents for the first four tricks. So we will introduce them simply and mainly focus on our proposed BNNeck and center loss. Fig. 2b shows training strategies and model architecture.

III-A Warmup Learning Rate

Learning rate has a great effect on the performance of a ReID model. Standard baseline is initially trained with a large and constant learning rate. In [40], a warmup strategy was applied to bootstrap the network for enhanced performance. In practice, we spend 10 epochs, thereby linearly increasing the learning rate from $3.5\times 10^{-5}$ to $3.5\times 10^{-4}$. The learning rate is decayed to $3.5\times 10^{-5}$ and $3.5\times 10^{-6}$ at 40th and 70th epochs, respectively. The learning rate $lr(t)$ at epoch $t$ is compute as follows:

[TABLE]

III-B Random Erasing Augmentation

In person ReID, persons in the images are sometimes occluded by other objects. To address the occlusion problem and improve the generalization capability of ReID models, Zhong et al. [41] proposed a new data augmentation approach, namely, random erasing augmentation (REA). REA randomly masks a rectangle region of the training image with a manually set probability $p$. All hyper-parameters are set same with [41] and [1]. Some examples are shown in Fig. 3.

III-C Label Smoothing

A basic baseline in person ReID is the IDE [8] network, whose last layer outputs the ID prediction logits of images. Given an image, we denote $y$ as truth ID label and $p_{i}$ as ID prediction logits of class $i$. The ID loss is computed as follows:

[TABLE]

However, in person ReID, person IDs of the testing set do not appear in the training set. So it is important to prevent from overfitting training IDs for the ReID model. A widely used technique to prevent overfitting for a classification task is Label smoothing (LS) proposed in [42]. The construction of $q_{i}$ is changed to:

[TABLE]

where $\varepsilon$ is a constant to encourage the model to be less confident on the training set. In this study, $\varepsilon$ is set to be $0.1$. When the training set is not large, LS can significantly improve the model performance.

III-D Last Stride

A high spatial resolution always enriches feature granularity. In PCB[5], the last spatial down-sampling operation of the backbone network is removed to enlarge the spatial size of the feature map. For convenience, the last spatial down-sampling operation in the backbone network is denoted as the last stride. The last stride equals to 2 for ResNet50 backbone. When fed into an image with $256\times 128$ size, it outputs a feature map with a spatial size of $8\times 4$. If last stride is changed from 2 to 1, then we can obtain a feature map with increased spatial size ($16\times 8$). This manipulation only slightly increases the computation cost and does not involve extra training parameters. However, an increased spatial resolution brings significant improvement.

III-E BNNeck

As shown in Fig. 4(a), many state-of-the-art methods combined ID and triplet losses to constrain the same feature $f$. Combining these two losses always let the model achieve better performance. However, the better performance let us ignore that the targets of these two losses are inconsistent in the embedding space.

Fig. 5(a) presents that ID loss constructs several hyperplanes to separate the embedding space into different subspaces. The features of each class are distributed affinely in different subspaces. So ,cosine distance is more suitable than Euclidean distance for the model optimized by ID loss in the inference stage. However, triplet loss is computed by Euclidean distance and enhances intra-class compactness and inter-class separability in the Euclidean space. As shown in 5(b), The features of triplet loss appear a cluster distribution. If we use both losses to optimize a feature space simultaneously, then their goals may be inconsistent. During training, a possible problem is that one loss is reduced, whereas the other loss oscillates or even increases, as shown in Fig.7. Finally, triplet loss may influence the clear decision surfaces of ID loss, and ID loss may reduce the intra-class compactness of triplet loss. The feature distribution is tadpole shaped. Therefore, directly combining these two losses can boost the performance, but it is not the best way.

Xiong et al. [9] added a BN layer [43] iongbetween feature and ID loss, which is same as Fig. 8(d). The authors claimed that the BN layer overcomes the overfitting and boosts the performance of IDE baseline. However, we consider that the BN layer can smoothen the feature distribution in the embedding space. For ID loss (Fig. 5(a)), the BN layer will enhance the intra-class compactness. The BN layer can improve the performance of ID loss because the features close to the affine center lack clear decision surfaces and are difficult to distinguish. Nevertheless, such layer increases the cluster radius of intra-class feature for triplet loss. Thus, the decision surfaces of 5(e)(f) are stricter than those of Fig. 5(b)(c).

To overcome this problem, we design a structure, namely, BNNeck, as shown in Fig. 4(b). BNNeck adds a BN layer after features and before classifier FC layers. The BN and FC layers are initialized through Kaiming initialization proposed in [44]. The feature before the BN layer is denoted as $f_{t}$. We let $f_{t}$ pass through a BN layer to acquire the feature $f_{i}$. In the training stage, $f_{t}$ and $f_{i}$ are used to compute triplet and ID losses, respectively. Fig. 4(g) shows that $f_{t}$ not only can keep a compact distribution from but also acquires ID knowledge from ID loss. Affected by the BN layer and ID loss, the distribution of $f_{i}$ is tadpole shaped. In comparison with 4(c), $f_{i}$ has clear decision surfaces because of the weaker influence of the triple loss. Additional details are introduced in Section IV-C.

In the inference stage, we select $f_{i}$ to perform the person ReID task. Cosine distance metric can achieve better performance than Euclidean distance metric. Experimental results in Table. I show that BNNeck can improve the performance of the ReID model by a large margin.

III-F Center Loss

Person ReID is always regarded as a retrieval/ranking task. The evaluation protocols, i.e. CMC curve and mAP, are determined by the ranking results but ignore the clustering effect. However, for some ReID applications, such as tracking task, an important step is to decide on a distance threshold to separate positive and negative objects. As shown in Fig. 6, two cases can acquire the same ranking results, but probe2 is easy for the tracking task because of its intra-class compactness of positive pairs.

Focusing on relative distance, triplet loss is computed as:

[TABLE]

where $d_{p}$ and $d_{n}$ are feature distances of positive and negative pairs. $\alpha$ is the margin of triplet loss, and $[z]_{+}$ equals $max(z,0)$. In this study, $\alpha$ is set to $0.3$. However, the triplet loss only considers the difference between $d_{p}$ and $d_{n}$ and ignores their absolute values. For instance, when $d_{p}=0.3$ and $d_{n}=0.5$, the triplet loss is $0.1$. For another case, when $d_{p}=1.3$ and $d_{n}=1.5$, the triplet loss also is $0.1$. Triplet loss is determined by two randomly sampled person IDs. Ensuring that $d_{p}<d_{n}$ in the entire training dataset is difficult. In addition, intra-class compactness is ignored.

To compensate for the drawbacks of the triplet loss, we involve center loss [45] intraining, simultaneously learns a center for deep features of each class and penalizes the distances between the deep features and their corresponding class centers. The center loss function is formulated as follows:

[TABLE]

where $y_{j}$ is the label of the $j$th image in a mini-batch. $\boldsymbol{c}_{y_{j}}$ denotes the $y_{i}$th class center of deep features, and $B$ is the batch size number. The formulation effectively characterizes the intra-class variations. Minimizing center loss increases intra-class compactness. Our model includes three losses as follows:

[TABLE]

where $\beta$ is the balanced weight of center loss. In our baseline, $\beta$ is set to be $0.0005$.

IV Experiment

IV-A Datasets

We evaluate our models on Market1501 [46] and DukeMTMC-reID [47] datasets, because both datasets are widely used and large scale. Following the previous works, we use rank-1 accuracy and mAP for evaluation on both datasets.

Market1501 contains 32,217 images of 1,501 labeled persons of six camera views. The training set has 12,936 images from 751 identities, and the testing set has 19,732 images from 750 identities. In testing, 3,368 hand-drawn images from 750 identities are used as queries to retrieve the matching persons in the database. Single-query evaluation is used in this study.

DukeMTMC-reID is a new large-scale person ReID dataset and collects 36,411 images from 1,404 identities of eight camera views. The training set has 16,522 images from 702 identities, and the testing set has 19,889 images from other 702 identities. Single-query evaluation is used in this study.

IV-B Influences of Each Trick

Baseline-S only reaches 87.7% and 79.7% rank-1 accuracies on Market1501 and DukeMTMC-reID, respectively. The performance of standard baseline is similar to most baselines reported in other papers. Warmup strategy, REA, LS, stride change, BNNeck, and center loss are individually added to the model training process. The designed BNNeck boosts performance to a greater extent than other tricks, especially on DukeMTMC-reID. Finally, the baseline acquires 94.5% rank-1 accuracy and 85.9% mAP on Market1501 with these training tricks. On DukeMTMC-reID, the baseline achieves 86.4% rank-1 and 76.4% mAP accuracy . Thus, these training tricks boost the performance of the standard baseline by over 10% mAP. In order to achieve such improvement, our strong baseline only involves an extra BN layer and do not increase training time.

We also explore the the effectiveness of these tricks for cross domain ReID, the results can be present in II. In overall, most of these tricks apart from REA are also effective for cross domain ReID. We infer that by REA masking the regions of training images, the model learns additional knowledge in the source domain and performs poorly in the target domain.

IV-C Analysis of BNNeck

IV-C1 Different neck structures

To discuss the effectiveness of our BNNeck, we design several different neck structures, as shown as Fig. 8. In addition, some ablation studies also are analysed in Table III. Neck3 outperforms Neck1 and Neck2. In addition, BNNeck2 is worse than Neck2, but BNNeck1 is better than Neck1. Our BNNeck achieves the best performance on two benchmarks. In summary, we present the following observations/conclusions.

1. Without the BN layer, integrating ID and triplet losses is better than only using one loss.
2. The BN layer is effective for ID loss but is invalid for triplet loss.
3. Our BNNeck that sets triplet loss before the BN layer is a reasonable neck structure.

IV-C2 Inconsistency between ID loss and Triplet loss

To verify that ID and triplet losses are inconsistent in the same feature space, we train the models with Neck3, BNNeck3, and our proposed BNNeck. Fig. 8 shows that these three neck structures use ID and triplet losses to optimize the same feature. Fig. 7 presents the training loss curves of 500 iterations. In Fig. 7a and 7d, the triplet loss initially increases and then decays in the loss curves marked by black ovals, showing a clear confrontation between triplet and ID losses. In comparison with Neck3, BNNeck3 adds a BN layer after f. In Figs. 7b and 7e, the BN layer weakens but does not eliminate the inconsistency. However, for BNNeck in Figs. 7c and 7f, the inconsistency is suppressed, and the triplet loss curves are smoothened. In conclusion, the BN layer can weaken the inconsistency between the losses, and separating them into two different feature spaces is important.

IV-C3 Visualization of feature distribution

To analyze the distribution of the different features in Fig. 8, we train models in MNIST dataset. The visualization has considerable noise because the number of person IDs on ReID benchmark is large, and the number of images from each person ID is small. By contrast, MNIST only has 10 categories, and each category consists of thousands of samples, making the feature distribution clear and robust. Fig. 9 shows the results. ID and triplet losses have two different feature distributions. When integrating these two losses in Fig. 9c, the clustered distribution is stretched to be tadpole shaped. The distributions of (d$sim$f) are more gaussian than those of (a$sim$c) because of the BN effect. Figs. 9g and 9h show that our BNNeck separates triplet and ID losses into two different feature spaces. The feature distribution of triplet loss remains clustered, and that of ID loss has clear decision surfaces similar to Figs. 9a and 9b.

We summarize our conclusions or observations as follows:

1. The feature distributions of ID and triplet losses are affined and clustered, i.e., they are inconsistent.
2. The feature distribution of ID+Triplet loss is tadpole shaped.
3. The BN layer can smoothen/normalize the feature distribution and enhance the intra-class compactness for ID loss but reduce it for triplet loss.
4. We separate triplet and ID losses into two different and suitable feature spaces.

IV-C4 Two feature space of BNNeck

Although the results on MNIST in Fig. 9 can efficiently support our conclusion, image classification and person ReID are two different tasks. We perform statistical analysis on the norm distribution of $f_{t}$ and $f_{i}$ in BNNeck on Market1501 dataset. The mean value $\mu$ and standard deviation $\sigma$ of feature norm are calculated. To analyze the separability of feature distribution, Coefficient of Variation $C.V.=\mu/\sigma$ is also present. As shown in Fig. 10, $f_{i}$ and $f_{t}$ are distributed differently in the feature space. $f_{t}$ is compactly and gaussian distributed in an annular space because it is directly optimized by triplet loss. However, we consider $f_{i}$ as a tadpole-shaped distribution because ID loss stretches intra-class distribution. The maximum value of $f_{i}$ is 48.70, whereas the $\mu$ is 18.62. $C.V.$ of $f_{t}$ is 0.043, but $C.V.$ of $f_{i}$ reaches 0.98, which demonstrates that $f_{i}$ is distributed more discretely than $f_{t}$. In conclusion, BNNeck provide two different and suitable feature spaces for triplet loss and ID loss.

IV-C5 Metric space for BNNeck

We evaluate the performance of two different features ($f_{t}$ and $f_{i}$) with Euclidean and cosine distance metrics. All models are trained without center loss in Table. IV. We observe that cosine distance metric performs better than Euclidean distance metric for $f_{t}$. As ID loss directly constrains the features followed the BN layer, $f_{i}$ can be clearly separated by several hyperplanes. The cosine distance can measure the angle between feature vectors; thus, cosine distance metric is more suitable than Euclidean distance metric for $f_{i}$. However, $f_{t}$ is simultaneously close to triplet loss and constrained by ID loss. The two types of metrics achieve similar performance for $f_{t}$.

Overall, BNNeck significantly improve the performance of ReID models. We select $f_{i}$ with cosine distance metric to perform the retrieval in the inference stage.

IV-D Analysis of Center loss

We discuss the influence of center loss on intra-class compactness. We consider that average intra-class distance cannot fully represent the intra-class compactness because it ignores inter-class distance. For convenience, the average intra-class and inter-class distances are denoted as $D_{p}$ and $D_{n}$, respectively. Inspired by [48], the ratio of $D_{p}$ to $D_{n}$ is used to measure the clustering effect of feature distribution. The ratio is computed as $R=D_{p}/D_{n}$. We set $\beta$ to different values and evaluate rank-1, mAP, and $R$ of the models. Table V presents the results.

For the feature $f_{t}$ constrained directly by center loss, $R$ decreases as $\beta$ increases. With $\beta$ increasing from 0 to 0.5, $R$ is reduced from 0.407 to 0.311 on Market1501 and from 0.424 to 0.363 on DukeMTMC-reID. Hence, center loss can improve intra-class compactness and inter-class separability, thereby bringing a clear boundary between positive and negative samples. When $\beta$ is set to 0.5, $f_{t}$ can acquire the best clustering effect but obtains the worse rank-1 and mAP accuracies. However, the BN layer destroys such clustering effect. For feature $f_{i}$, the value of $R$ is almost not influenced by $\beta$. On the basis of these observations, we arrive at the following conclusions: (1) Center loss boosts intra-class compactness and inter-class separability. (2) The BN layer can destroy the effect of center loss. (3) Increasing the weight of center loss may reduce ranking performance.

IV-E Comparison to Other Baselines

We compare our strong baseline with other effective baselines, such as IDE [8], TriNet [49], AWTL [50] and PCB [5]. PCB is a part-based baseline for person ReID. Table VI presents the performance of these baselines. The experimental results show that our baseline outperforms IDE, TriNet, and AWTL by a large margin. PCB integrates multi-part features and GP uses effective tricks, and both of them achieves great performance. However, our baseline surpasses them by over 7.1% mAP on both datasets. To our best knowledge, our baseline is the strongest baseline.

IV-F Comparison to State-of-the-Arts

We compare our strong baseline with some state-of-the-art methods in Table. VII. All methods have been divided into different types. Pyramid[51] achieves surprising performance on two datasets, but it concatenates 21 local features of different scales. When only the global feature is utilized, Pyramid obtains 92.8% rank-1 accuracy and 82.1% mAP on Market1501. Our strong baseline can reach 94.5% rank-1 accuracy and 85.9% mAP on Market1501. BFE[52] obtains similar performance to our strong baseline, but it combines features of two branches. Among all methods that only use global features, our strong baseline outperforms AWTL[50] by more than 10% mAP on both Market1501 and DukeMTMC-reID. To our best knowledge, our baseline achieves the best performance when only global features are used. Our method, which does not require human semantic information, local features or attention modules, is simpler than other state-of-the-art methods.

IV-G Baseline Meets State-of-the-Arts

We reproduce some popular state-of-the-art methods with our strong baseline. Given numerous outstanding methods are available, we cannot try all of them and select only several typical models such as $k$-reciprocal re-ranking[36], PCB[5], AligedReID++[3], CamStyle[31], and MGN[16]. For a fair comparison, we use the same losses as the paper reported to train the models. For instance, AlignedReID++ only uses ID and triplet losses, and we do not use center loss to reproduce it. However, as $k$-reciprocal re-ranking is a post-processing method of global features, three losses are used to improve its performance. Table VIII shows the details and results, wherein the values in parentheses are the results reported by authors in their papers. In addition, we present the performance of the baselines (with BNNeck) trained by different losses as a reference.

Our baseline boosts the performance of $k$-reciprocal re-ranking, PCB, AligedReID++, and CamStyle by a large margin. The mAP of k-reciprocal re-ranking achieves +30.6% on Market1501, demonstrating that the performance of baselines is important for methods. In addition, our MGN achieves similar performance to [16] because its accuracies are too high to improve, and [16] uses BNNeck1 structure. Integrating multiple part features can reduce the effect of global features and limit the effectiveness of baselines for PCB and MGN. However, PCB and MGN still obtain better performance than Baseline2, i.e., part-based methods are effective for our baseline.

However, CamStyle(Our) outperforms CamStyle [31] but not Baseline1. Our baseline can be a strong baseline for the ReID community because it can boost the performance of some methods, and other methods based on it may be ineffective. To some extent, our baseline efficiently filters effective methods.

IV-H Performance of Different Backbones

All aforementioned models apply ResNet50 as backbones for clear ablation studies and comparison with other methods.

Models with different backbones, such as ResNet, SeResNet, SeResNeXt, and IBNNet, are evaluated because backbones have a great influence on their performance. As shown in Table IX, deep and large backbones can achieve high performance. For example, ResNet101 outperforms ResNet18 by 2.8% and 9.3% in Rank-1 and mAP accuracy on Market1501, respectively. In addition, the channel attention of SeNet and group convolution of ResNeXt can enhance the performance by a slight margin. IBN-Net50 [54], which replaces the BN layers with instance BN layers for ResNet50, is also effective for our baseline. Specifically, IBN-Net50-a is suitable for standard ReID task and obtains 95.0% and 90.1% rank-1 accuracies on Market1501 and DukeMTMC-reID, respectively. However, IBN-Net50-b achieves 50.1% rank-1 and 29.8% mAP for Market1501$\to$DukeMTMCReID (M$\to$D) and 61.7% rank-1 and 32.0% mAP DukeMTMCReID1501$\to$Market1501 (D$\to$M).

For comparison, IBN-Net50-a achieves 40.0% rank-1 and 25.1% mAP for M$\to$D and 52.9% rank-1 and 25.1% mAP (D$\to$M). In conclusion, IBN-Net-a and IBN-Net50-b are suitable for the same domain task and the cross-domain task, respectively.

V Conclusions and Outlooks

In this study, we propose a strong baseline for person ReID with only adding an extra BN layer for standard baseline. Our strong baseline achieves 94.5% rank-1 accuracy and 85.9% mAP on Market1501. To our best knowledge, this result is the best performance achieved by the global features of a single backbone. We evaluate each trick of our baseline on same- and cross-domain ReID tasks. In addition, some state-of-the-art methods can be effectively extended on our baseline. We hope that this work can promote ReID research in the academia and industry.

We observe the inconsistency between ID and triplet losses in previous ReID baselines. To address this problem, we propose a BNNeck to separate both losses into two different feature spaces. Extended experiments show that the BN layer can enhance and reduce the intra-class compactness for ID and triplet losses, respectively. Furthermore, ID loss is suitable for optimizing the feature.

We emphasize that the evaluation of ReID task ignores the clustering effect of representation features. However, the clustering effect is important to some ReID applications, such as tracking task wherein an important step is deciding on a distance threshold to separate positive and negative objects. A simple way to address this problem is using center loss to train the model. Center loss can boost the clustering effect of features, but may reduce the ranking performance of ReID models.

In the future, we will explore additional tricks and effective methods based on this strong baseline. In comparison with face recognition, person ReID still has room for further exploration. In addition, some confusions remain, such as why REA reduces the cross-domain performance in our baseline. Points wherein the conclusion is unclear are worth researching.

Acknowledgment

This research was funded by the National Natural Science Foundation of China under Grant 61633019, the Science Foundation of Chinese Aerospace Industry under Grant JCKY2018204B053 and the Autonomous Research Project of the State Key Laboratory of Industrial Control Technology, China (Grant No. ICT1917).