Person Re-identification with Bias-controlled Adversarial Training

TL;DR

This paper introduces a Bias-controlled Adversarial framework (BCA) for person re-identification that effectively manages biases like pose, body part, and camera view, improving accuracy across various scenarios.

Contribution

It presents a novel adversarial training approach that controls bias features, enhancing person re-ID performance beyond existing methods.

Findings

Improves re-ID accuracy on multiple benchmarks.

Effective in full and partial view scenarios.

Outperforms state-of-the-art methods.

Abstract

Inspired by the effectiveness of adversarial training in the area of Generative Adversarial Networks we present a new approach for learning feature representations in person re-identification. We investigate different types of bias that typically occur in re-ID scenarios, i.e., pose, body part and camera view, and propose a general approach to address them. We introduce an adversarial strategy for controlling bias, named Bias-controlled Adversarial framework (BCA), with two complementary branches to reduce or to enhance bias-related features. The results and comparison to the state of the art on different benchmarks show that our framework is an effective strategy for person re-identification. The performance improvements are in both full and partial views of persons.

Tables

Table 1. Table 1: Analysis of re-ID and bias classification task under different values of λ D ​ b subscript 𝜆 𝐷 𝑏 \lambda_{Db} parameter for either reducing (R) or enhancing (E) bias features on CUHK03 (labeled split and new protocol), Market-1501 (single query), and CropCUHK03. Re-ID performance is measured in terms of Rank1 and mAP (mean average precision); Bias classification task is evaluated in terms of accuracy (acc).

	Pose			Camera			BodyPart
	CUHK03			Market-1501			CropCuhk03
	re-ID		Bias	re-ID		Bias	re-ID		Bias
	Rank1	mAP	acc	Rank1	mAP	acc	Rank1	mAP	acc
Bas	60.7	72	.78	87	72.9	.44	39.6	52.6	.56
NoBias	63.5	74	-	88.6	75.3	-	60.5	69.8	-
$-{\lambda }_{Db}$	Bias-reducing (R) $\left({\lambda }_{Dr}=1\right)$
$.1$	26.5	39.6	.50	79.2	64.4	.22	15.6	26	.53
$.05$	56.2	68.3	.51	83.6	68.7	.22	26.1	38.4	.53
$.01$	62.5	73.3	.62	87.2	72.9	.27	40.6	53.6	.54
$.005$	61.7	72.9	.71	86.8	73.2	.31	38.8	51.8	.54
${\lambda }_{Db}$	Bias-enhancing (E) $\left({\lambda }_{Dr}=0\right)$
$1$	.2	1.3	1	.0	.1	.82	.2	1.1	.66
${\lambda }_{Db}$	Bias-enhancing (E) $\left({\lambda }_{Dr}=1\right)$
$.005$	60.7	71.9	.77	87.8	72.9	.44	40.3	53.2	.56
$.01$	61.2	72.4	.74	88.5	73.8	.43	39.9	52.7	.56
$.05$	62.4	73.2	.69	88.2	74.5	.43	36.1	49	.55
$.1$	61.4	72.4	.62	87.4	72.9	.32	21.9	33.5	.53
R+E	65.1	75.3	-	89.9	77.1	-	42.1	54.7	-

Table 2. Table 2: Comparison results in Market-1501 and Duke-reID. Our method with controlled pose and camera bias R+E pose+cam outperforms all GANs and adversarial learning methods. Yet, when combined with re-ranking R+E(RR+) pose+cam the performance further increases.

	MARKET-1501		DukeMTCM
Method	Rank1	mAP	Rank 1	mAP
LOMO+XQDA [38]	-	-	30.8	17
MSCAN [11]	80.3	57.5	-	-
Spindle [16]	76.9	-	-	-
SVDNet [39]	82.3	62.1	76.7	56.8
PCB [35]	93.8	81.6	83.3	69.2
GAN
DCGAN [27]	83.9	66.1	67.7	47.1
Pose-transfer [2]	87.6	68.9	78.5	56.9
CamStyle [5]	88.1	68.7	75.3	53.5
CamStyle(RR+) [5]	89.5	71.5	78.3	57.6
Pose-norm [37]	89.4	72.6	73.5	53.2
Attention
ACCN (D+) [7]	85.9	66.9	76.8	59.2
ACCN (D+RR+) [7]	87.7	83.0	-	-
HA-CNN [6]	91.2	75.7	80.5	63.8
DuATM (A+) [8]	91.4	76.6	81.8	64.6
Mancs [36]	93.1	82.3	84.9	71.8
Background bias
E. Backgr-Bias [13]	80.5	-	-	-
E. Backgr-Bias(D+) [13]	81.2	-	-	-
MaskG[10]	83.8	74.3	-	-
Adversarial learning
DR-GAN (siam) [26]	86.7	73.2	76.9	60.2
FD-GAN [25]	90.5	77.7	80.0	64.5
Our method
baseline	87.0	72.9	78.4	63.6
R+E pose	89.9	77.3	80.9	67.2
R+E cam	89.9	77.1	80.1	67.0
R+E pose+cam	90.4	77.9	82.2	68.9
R+E (RR+) pose+cam	93.1	89.3	85.2	74.8

Table 3. Table 3: Comparative evaluation in CUHK03. Our framework R+E pose and with re-ranking incorporated R+E (RR+) pose report competitive results on this benchmark.

	CUHK03 (L)		CUHK03 (D)
Method	Rank1	mAP	Rank1	mAP
LOMO+XQDA [38]	14.8	13.6	12.8	11.5
SVDNet [39]	-	-	41.5	37.6
PCB [35]	-	-	63.7	57.5
GAN
Pose-transfer [2]	45.1	42	41.6	38.7
Attention
ACCN (D+) [7]	81.9	81.61	79.1	78.4
HA-CNN [6]	44.4	41	41.7	38.6
Mancs [36]	69	63.9	65.5	60.5
Background bias	-	-
MaskG [10]	50.1	50.2	46.7	46.9
Our method	-	-
Baseline	60.7	72	58	69.2
R+E pose	65.1	75.3	62.3	72.8
R+E (RR+) pose	75.5	83.9	73.4	81.5

Table 4. Table 4: Comparative evaluation in CropCUHK03. Our approach outperforms the other methods evaluated on this dataset.

	CropCUHK03
Method	R1	mAP
Res50	9.6	9.3
Res50 (RR+)	11.6	12.8
Res50+XQDA	16.6	14.5
Res50+XQDA (RR+)	18.6	20
PCB bas.	25.9	16.5
PartialMatchNet [15]	31.3	21.2
Baseline [31]	39.6	52.6
R+E body part	42.4	55.2
R+E pose	42.1	54.7
R+E body part+pose	43.5	56.3
R+E body (RR+) part+pose	54.3	65.6

Equations

$$\begin{array}{rl}\underset{W}{\min }\underset{\Phi }{\max }L(G,D)& ={\mathbb{E}}_{x\sim p_r(x)}[\log D_{\Phi }(x)]\\ & +{\mathbb{E}}_{z\sim p_z(z)}[\log (1-D_{\Phi }(G_W(z))]\end{array}$$

$$\begin{array}{r}{\mathcal{L}}_{adv}=-{\mathbb{E}}_{x\sim p_r(x)}[\log D(x)]-{\mathbb{E}}_{x\sim g(x)}[\log (1-D(x))]\end{array}$$

$$\begin{array}{rl}{\mathcal{L}}_{adv}& =\\ & \sum _{i=1}^n-{\mathbb{E}}_{x\sim p_r(x)}[\log D_i(x)]-{\mathbb{E}}_{x\sim g(x)}[\log (1-D_i(x))]\end{array}$$

$$\begin{array}{rl}{\mathcal{L}}_{adv}(f)=& -{\mathbb{E}}_{x\sim X_s}[\log D(f(x))]\\ & -{\mathbb{E}}_{x\sim X_t}[\log (1-D(f(x))]\end{array}$$

$$\begin{array}{r}{\mathcal{L}}_{adv}=||{\mathbb{E}}_{x\sim p_r(x)}f(x)-{\mathbb{E}}_{z\sim g(z)}f(G(z))|{|}_2^2\end{array}$$

$$\begin{array}{r}\mathcal{L}(X)=\sum _{x_a\in X}[m+\stackrel{hardpositive}{\overbrace{\underset{x_p}{\max }||x_a-x_p|{|}^2}}-\stackrel{hardnegative}{\overbrace{\underset{x_n}{\min }||x_a-x_n|{|}^2}}{]}_{+}\end{array}$$

$$\begin{array}{rl}{\mathcal{L}}_{Dr}(X)& =\\ & \sum _{a_r\in X}[m+\stackrel{hardpositive}{\overbrace{\underset{p_r}{\max }||{\mathbb{E}}_{a_r\sim p(a_r)}f(a_r)-{\mathbb{E}}_{p_r\sim p(p_r)}f(p_r)|{|}^2}}\\ & -\stackrel{hardnegative}{\overbrace{\underset{n_r}{\min }||{\mathbb{E}}_{a_r\sim p(a_r)}f(a_r)-{\mathbb{E}}_{n_r\sim p(n_r)}f(n_r)}}|{|}^2{]}_{+}\end{array}$$

$$\begin{array}{rl}{\mathcal{L}}_{Db}(X)& =\\ & \sum _{a_b\in X}[m+\stackrel{easypositive}{\overbrace{\underset{p_b}{\min }||{\mathbb{E}}_{a_b\sim p(a_b)}f(a_b)-{\mathbb{E}}_{p_b\sim p(p_b)}f(p_b)|{|}^2}}\\ & -\stackrel{easynegative}{\overbrace{\underset{n_b}{\max }||{\mathbb{E}}_{a_b\sim p(a_b)}f(a_b)-{\mathbb{E}}_{n_b\sim p(n_b)}f(n_b)}}|{|}^2{]}_{+}\end{array}$$

$${\mathcal{L}}_{adv}(X)={\lambda }_{Dr}\cdot {\mathcal{L}}_{D_r}(f_{\circ }(X))\circ {\lambda }_{Db}\cdot {\mathcal{L}}_{D_b}(f_{\circ }(X))$$

Full text

1 Introduction

Convolutional Neural Networks (CNNs) with recent innovations such as GANs [2, 3, 4, 5] and attention based models [6, 7, 8] have fostered rapid development in human re-identification obtaining impressive results. However, these re-ID methods rely on CNN features which are susceptible to bias that the training data suffers from. Bias occurs when similarity between two data samples in re-ID problem is due to the same pose, body part or camera view, rather than to the ID-related cues. For example, [9] recently demonstrated that popular CNN architectures, i.e., ResNet-50 focus on local textures rather than global shapes for recognizing objects in ImageNet. It was also observed [10] that the background, i.e., camera view significantly affects feature representations in person re-ID when training on existing academic datasets. Other works [2, 14] deal with pose bias by generating multiple synthetic poses for training [2] or by leveraging body part cues [14]. In contrast to all these methods that focus on one specific bias only, we propose a general approach that can address different types of bias without additional training data.

In figure 1 we show examples of how pose, camera view, and body part affect the re-id rankings. These are top ranked returns given by a well performing TriNet [31] on CUHK03 [1], Market-1501 [23] and CropCUHK03 [15] with partial views. Frames with different IDs are marked by a red x above the frame. The top row shows the correct ID ranked $4$th after three different subjects in the same pose as the query; the second row illustrates camera view bias, where the correct ID is ranked next to similar subjects appearing in the same background as the query; finally, the bottom row shows body part bias where the same body crop/part is ranked higher than the correct ID, which is ranked $13$th. Note that in person re-ID benchmarks the positive examples with the same bias are not included when calculating the performance score. This, however, can also be considered a certain type of bias as it does not exactly correspond to the practical scenario. Nonetheless, in re-ID settings, the bias shows with the top-ranked negative examples affecting re-ID performance as illustrated in figure 1. Its influence can be reduced by large and diverse training data. However, in existing benchmarks, the small number of training examples often reflects specific settings from data collection. In this paper, we present an adversarial framework for controlling the bias in re-ID representation. Specifically, our model includes a feature mapping combined with an identity and a bias discriminators, that exploit examples with the same ID or the same bias. Our framework contains two specialized branches: one for bias-reducing in the image representations, e.g., pose, body part, background, thus focusing on ID features, e.g., clothes details; and one for bias-enhancing required for correct separation from ID features. The two branches generate descriptors containing complementary information to produce the final image representation. The contributions of this work are the following:

1. We introduce the idea of bias discriminator that helps to extract robust descriptors in re-ID; specifically, we propose bias-controlled adversarial training with two network branches specialized in revealing bias-related and bias-unrelated structures in data, which generalizes existing adversarial solutions [26, 25] to other types of bias in human re-identification.
2. This is the first work that provides quantitative analysis on how bias factors, such as the pose, camera view and body parts affect re-ID performance. 3) We demonstrate experimentally that the two proposed branches effectively reduce distracting features and enhance essential re-ID cues. We show the importance of including both bias and ID features in the re-ID task leading to competitive results in several benchmarks with full and partial views. The two branches generate descriptors containing complementary information to produce the final image representation.

2 Related work

In this section, we review closely related methods and discuss the differences to our approach.

Background and Pose Bias. Despite many methods [11, 16, 14, 10] have proposed to leverage different regions/pixels to obtain descriptors robust to pose and background, the bias problem has not been explicitly analyzed and addressed. The only work investigating the influence of background is [13] with the support of foreground masks. Instead, our framework relies on available annotations and low inference complexity compared to pixel based predictions.

GANs based methods [27, 14, 2, 5, 3, 4] can potentially alleviate the bias problem by generating many synthetic samples with diversity, for example, in different poses [14, 2], camera views [5], and domains [3, 4]. However, none of these methods provides quantitative analyses of the bias problem in re-ID.

Adversarial Training in Recognition. Adversarial learning was previously adopted for face and person re-ID [26, 25] with similar goal of disentangling representations from biased features. However, unlike our work, [25] addressed the pose bias only with a specific solution relying on a pose landmark map as input for training. Furthermore, an unsupervised method [26] learns ID related and unrelated features for face identification, but its generalization to articulated pedestrian bodies is not straightforward. Our model relies on training data from standard datasets only and leads to a better performance than [26], as shown in the experimental results.

Adversarial Training in Domain Adaptation from labelled source to unlabelled target has been proved to be particularly effective. According to the domain adaptation principles [17], predictions should be made on features that can not discriminate between the source and the target domains. Thus [18, 19, 20] incorporate adversarial training to promote features that are discriminant for the main task in the source domain and non-discriminant with respect to the shift between the domains. Likewise, we adopt adversarial training to extract features that are discriminant for the re-ID task, and non-discriminant with respect to the bias classes.

Partial re-ID is an emerging problem introduced in [22], which focused on recognizing or matching identities of people from frames containing only parts of human bodies. A local-to-local matching based on small patches and global-to-local with a sliding window search for partial templates was used in [22]. Another approach [28] reconstructs missing channels in the query feature maps from full observations in the gallery maps. However, [22] was validated on small datasets, i.e., Partial Reid, i-LIDS, and Caviar, and may not scale well to real scenarios due to $On^{4}$ complexity during inference time, while [28] relies on the assumption that gallery images always contain full body of persons. Recently, [15] proposed a solution based on alignment and hallucination as well as a synthetic data generated from CUHK03 with partial views in both probe and gallery images. Similarly, in order to analyze partial view bias, we evaluate our approach on CropCUHK03 dataset [15] and compare with their proposed method.

3 Adversarial training

In this section, first we shortly summarise the main concepts our work is based on, and then present the proposed bias-controlled adversarial framework.

3.1 Preliminaries

Adversarial Training Framework. The goal of the generative adversarial framework [24] is to train a differentiable function $G_{W}$ parametrized by $W$, usually represented by deep neural networks, to generate fake samples as close as possible to the real data. $G_{W}$ is trained along with a discriminator $D_{\Phi}$, which estimates the likelihood that an input sample comes from the real distribution. $D_{\Phi}$ is adversarial to $G_{W}$ in the sense that by recognizing fake from real samples, $D_{\Phi}$ drives $G_{W}$ to generate more realistic examples.

This is known as $\min\max$ game with value function $L(G,D)$:

[TABLE]

${x}$ and $G_{W}(\cdot)$ are real and fake samples from input noise $z$, respectively. It results in the following adversarial loss:

[TABLE]

Many variants of GANs have been proposed with modifications to the original framework making it a powerful and flexible tool, e.g.,  [12] introduced multiple discriminators:

[TABLE]

In domain adaptation problem, [20] replaces the generator with the mapping $f$ for making target samples $X_{t}$ indistinguishable from those in the source distribution $X_{s}$:

[TABLE]

A new objective, as an alternative to the binary cross entropy, was proposed in [21]. It is based on similarity between distributions of the generated and real data:

[TABLE]

Inspired by these methods, we 1) use multiple discriminators [12] to recognize samples with the same identity or bias; 2) instead of employing a generator, we train feature mappings as in [20] with controlled bias; 3) we train the feature mappings as [21] to directly match the corresponding feature vector of the correct sample, instead of using the typical cross entropy loss.

Triplet Loss. [30, 32, 33] Training samples are selected and arranged in triplets $x_{a},x_{p},x_{n}$, where $x_{a},x_{p}$ is the pair with the same identity therefore their distance is minimized and $x_{a},x_{n}$ is the pair with different identities therefore their distance is maximized.

Batch Hard [31] has lead to a superior performance compared to the triplet loss [32, 33]. Positives and negatives are paired in a batch specifically designed to include $C$ random classes, with randomly sampled $K$ examples of each class, thus resulting in a batch of $CK$ images. For each anchor in the batch the hardest positive and the hardest negative sample is selected when forming triplets. This leads to the following formulation of loss with mini-batch $X$ and margin $m$:

[TABLE]

3.2 Proposed model

We introduce our Bias-controlled Adversarial framework (BCA) with two main branches for bias-reducing (R) and bias-enhancing (E). The former suppresses features corresponding to the body part/pose/camera bias and selects ID features that are robust against their variations. The latter enhances features relevant for pose, camera or body part recognition. A overview of the training framework is presented in figure 2. Each branch includes two discriminators $D_{r}$ and $D_{b}$ and the feature mapping $f$. While the re-ID discriminator $D_{r}$ attempts the re-ID task, the bias discriminator $D_{b}$ drives the feature mapping $f$ to reduce/enhance the bias features by minimizing the proposed adversarial triplet loss. Both branches are trained with adversarial loss for re-ID and for bias discriminator as presented below.

Re-ID Adversarial loss. We define the re-identification objective $\mathcal{L}_{Dr}$ as an adversarial triplet loss, which is a reinterpretation of batch hard triplet loss in equation 6 including GANs equations 4 and 5:

[TABLE]

where parameters of the feature mapping $f$ are optimized so that the distribution of positive ID class samples $p(p_{r})$ matches the distribution of anchor ID class samples $p(a_{r})$, and also differs from the distribution of negative ID samples $p(n_{r})$.

Bias Adversarial loss. Similarly to equation 7, bias adversarial loss is defined as:

[TABLE]

In contrast to $\mathcal{L}_{D_{r}}$, training triplets $\{p_{b},a_{b},n_{b}\}$ for $\mathcal{L}_{D_{b}}$ are selected by considering the bias types and the easiest positive/negative samples as more meaningful to the bias term. In other words, for the re-ID task, it is more beneficial to include hard cases in the batch, e.g, same persons with a different pose, or similar pose of different persons. Instead, for the bias term, the easiest cases are those where the bias is more evident and therefore should be used for learning bias related characteristics. However, there are no specific constraints on the ID when pairing samples for the bias. Adversarial Triplet Loss. Our objective evaluates the loss for a mini-batch of samples $X$ arranged in triplets with the batch hard strategy. It involves the bias discriminator $D_{b}$, the ID discriminator $D_{r}$ and the mapping functions $f_{\circ}$:

[TABLE]

$\mathcal{L}_{adv}(X)$ incorporates re-ID adversarial loss $\mathcal{L}_{Dr}$ and bias adversarial loss $\mathcal{L}_{Db}$ for training $D_{r}$ and $D_{b}$. Note that the loss corresponds to both the bias-reducing and bias-enhancing branch, depending on the sign ($\circ$) between the two components. Maximizing $\mathcal{L}_{Db}$ loss (-) reduces features related to bias in mapping images to features by $f_{-}$, while minimizing $\mathcal{L}_{Db}$ (+) leads to bias enhancing in $f_{+}$. Parameters $\lambda_{Dr}$ and $\lambda_{Db}$ control the contributions from these two terms in each branch.

**Bias-reducing branch ** is obtained by setting sign ($\circ$) to ($-$) in Equation 9, thus $\mathcal{L}_{Db}$ is maximized, i.e., the distance between samples from the same bias class $a_{b},p_{b}$, regardless their ID is maximized, while distance between different bias class samples $a_{b},n_{b}$ is minimized. This forces $f_{-}$ to reduce feature components encoding the bias and focus on unbiased patterns.

Bias-enhancing branch. $\mathcal{L}_{Db}$ is minimized by setting sign ($\circ$) to ($+$) in Equation 9, i.e., the distance between $a_{b}$ and $p_{b}$ is minimized, and between $a_{b}$ and $n_{b}$ is maximized. The network rewards biased samples, therefore emphasizes feature components encoding the bias. As we show in the experiments, it provides a complementary mapping branch $f_{+}$ that improves the representation for re-ID task.

Re-ID loss $\mathcal{L}_{Dr}$ is used in training in the same way in both scenarios to reduce the distance between positive ID pairs and increase for the negative ones. In contrast, the bias loss $\mathcal{L}_{Db}$ differs for the two cases i.e., maximizing or minimizing the distance between positive bias pair for reducing and enhancing branches, respectively. Finally, during inference time the two embeddings - coming from the mappings $f_{-}$ and $f_{+}$ from the two independent branches - are concatenated to produce the final image representation.

4 Experimental results

In this section, we first describe datasets used for our evaluation and provide implementation details. We then analyze the effectiveness of our BCA framework for different types of bias. Finally, we report the performance on standard benchmarks and compare our approach against other methods on full and partial views.

4.1 Datasets and bias annotation

CUHK03 [1] includes $14,096$ images of $1,467$ pedestrians from 6 surveillance cameras. We adopt the new evaluation protocol from [34]. In this dataset we experiment with the pose bias considering two class labels, i.e., frontal and profile views, which also correspond to the two disjoint camera views.

**MARKET-1501 ** [23] contains $32,668$ annotated bounding boxes of $1,501$ identities. The images are acquired from six surveillance cameras including $5$ high-resolution cameras and $1$ low-resolution camera. In Market-1501 we investigate both the camera view and pose bias using the available camera annotation and the proposed three class pose labels, i.e., frontal, side and oblique.

DUKE [29] provides $36,411$ bounding boxes of $1,501$ identities. As in Market-1501, we analyze both the camera view and the pose bias from the existing camera annotation and our three pose classes.

CropCUHK03 [15] is a dataset derived from CUHK03 [1] by cropping partial views but maintaining the same number of individuals and frames. We use CropCUHK03 settings ${}_{s=0.5,o_{min}=.25}$ where $s$ is the fraction of area cropped from full labeled frame and $o_{min}$ is the minimum overlap between two views. The pose labels are frontal and side. In addition, we annotate each partial view with three body part labels, i.e., upper, central, bottom parts.

Note that in all the datasets class labels are uniformly distributed. Our pose annotation for Market-1501 and Duke as well as body part annotation for CropCUHK03 are available online111https://github.com/iodicesara/person-re-identification-with-bias-controlled-adversarial-training.

4.2 Implementation details

Baseline. We use an implementation of TriNet [31] as a baseline for comparison in our experiment. We set a batch size of $P\cdot K=64$ samples with $P=16$ randomly extracted persons IDs and $K=4$ instances for each person. We train the network with Adam optimizer, with a learning rate of $0.0003$ and linear decay to zero over $60$ epochs. As common practice during the training, we perform data augmentation by using random horizontal flips and re-scaling input images to $384\times 128$ pixels. Further implementation details can be found in [31].

Bias-reducing/enhancing branches. ResNet50 is used as a backbone of the discriminators $\{D_{b},D_{r}\}$ and the mapping $f$ in each of the two branches. In our implementation, these three components share the same weights within each of the ResNet50 branch. Each branch is trained independently and with the same hyper-parameters, optimizer and augmentation strategy as the baseline. In bias-reducing, we set $\lambda_{D_{b}}=-0.01$ for all types of bias. In bias-enhancing, we set $\lambda_{D_{b}}=0.05$ for pose bias and $\lambda_{D_{b}}=0.01$ for camera and body part bias. As a good practice, we kept consistent across different datasets the values of $\lambda_{D_{b}}$. During inference, mappings $f_{-}$ and $f_{+}$ output a concatenated descriptor of $2\times 2048$ components from the-second-to-last layers of ResNet50. Note that no bias annotation is required during inference time.

4.3 Evaluation of individual components

We evaluate the effectiveness of our BCA framework for different bias, i.e., pose, camera, and body part. We first demonstrate that bias-reducing and bias-enhancing branches effectively suppress and boost the bias features by evaluating the generated re-ID descriptors for bias classification tasks. Furthermore, we show the importance of bias related features and how they affect the representations that are essential for the re-ID task. Finally, we statistically analyze the occurrence of bias during evaluation and prove that our method is able to control it. The bias classification experiment shows the extent to which the performance of a strong TriNet baseline [31] is affected by the pose, camera and body parts. We extract features with pretrained mappings $f_{\circ}$ from bias-reducing (R) and bias-enhancing (E) branches on three different datasets. CUHK03 for front/side pose recognition; Market-1501 for six cameras; CropCUHK03 for top, central and bottom body parts. We use the extracted features as inputs to train a simple classification model based on a PReLU activation layer and a fully connected layer with a number of outputs corresponding to the bias class labels, i.e., $2$ for pose, $6$ for camera, and $3$ for body part. Note that while training the bias classifier the feature extracting backbone is fixed. Table 1 shows the results which we discuss in detail below.

Baseline. In order to measure the top performance the system can reach we evaluate the baseline disregarding biased pose, camera or body part samples. Specifically, during inference for each query we remove the samples from the gallery list that have the same bias label as the query and calculate the scores.

Note that, re-ID protocols already discard positives with the same camera view to make the task challenging and in this experiment we additionally exclude the negatives with the same bias. The results show that the re-ID performance is significantly affected by the bias as we observe Rank1 improvement in each of the benchmarks by $2.8\%$ on CUHK03, by $1.6\%$ on Market-1501, and by $20.9\%$ on CropCUHK03. The results also show that re-ID features from the baseline include information on specific bias as the bias classification accuracy is $0.78$, $0.44$ and $0.56$ for pose, camera and body parts, respectively.

Bias control. Next we demonstrate the ability of the proposed framework to control the bias. We train the two branches for different balance between the re-ID and the bias loss in equation 9, i.e., we vary the values of $\lambda_{Db}$ for pose, camera and body parts while keeping re-ID loss fixed $\lambda_{Dr}=1$. For example in table 1, by setting $\lambda_{Db}$ to negative weight of $-0.01$ i.e., the bias classification accuracy is reduced from $0.78$ to $0.62$ for the pose bias in CUHK03 and from $0.44$ to $0.27$ for the camera bias in Market-1501, with re-ID performance slightly increased. Further increasing negative $\lambda_{Db}$ deteriorates the re-ID performance. It is also evident that body part bias is more challenging to control than other types, however we are still able to reduce the bias classification accuracy in CropCUHK03 from $0.56$ to $0.54$ while increasing re-ID in both Rank1 and mAP metrics. Note that, re-ID performance is very sensitive to the value of $\lambda_{Db}$ and its behaviour is somewhat opposite in the two branches. Moreover, when re-ID loss $\lambda_{Dr}$ is deactivated and $\lambda_{Db}=1$, features from Bias-enhancing branch well recognizes all the bias types. The classification accuracy is $1$ for pose, $0.82$ for six cameras, and $0.66$ for top, central and bottom body parts. These suggest that the bias features in re-ID descriptors and features learned from the bias labels are complementary which motivates us to include two different branches for controlling the bias in person representation. The reducing one for disentangling the bias from re-ID features in order to obtain robust representation. The enhancing one for boosting features from the available bias annotation. Note that the importance of including bias related features was previously highlighted by other works which incorporate shape features from background masks [14] and body parts [10] into re-ID representations as related to essential cues for re-ID.

Our results further supports their observations.

Effect of bias on re-ID. An important observation under bias-reducing mode in table 1 is that by increasing the contribution from ${\lambda_{Db}=0.005}$ to ${\lambda_{Db}=0.1}$, Rank1 dramatically drops from $60.7\%$ to $26.5\%$ in CUHK03, from $87\%$ to $79.2\%$ in Market-1501 and from $39.6\%$ to $15.6\%$ with respect to the baseline. This suggests that pose, camera and body part features cannot be completely suppressed without affecting re-ID features. We found that effective bias reduction is with $\lambda_{Db}\leq 0.01$ resulting in an improvement from $60.7\%$ to $62.5\%$ in CUHK03 and $39.6\%$ to $40.6\%$ in CropCUHK03.

Another observation is that by emphasizing the opposite configuration, which enhances the bias features related to the annotation, Rank1 increases by nearly $2\%$ in CUHK03 ($\lambda_{Db}=0.05$) and by $1.5\%$ in Market-1501 ($\lambda_{Db}=0.01$). Finally, concatenating the embeddings derived from both branches results in a considerable boost in Rank1 in all three benchmarks, i.e., from $60.7\%$ to $65.1\%$ in CUHK03, from $87\%$ to $89.9\%$ in Market-1501 and from $39.6\%$ to $42.1\%$ (case: $R+E$). This further suggests that bias-reducing and bias-enhancing branches extract complementary features and form a better descriptor for re-ID when used jointly.

Statistical analysis of bias in retrieval. Ranked gallery samples returned by the re-ID system include both positive and negative examples. Re-ID evaluation protocols discard only the positives from the same camera, however, the negatives are also biased and affect the re-ID scores. We analyze the influence of pose and camera bias in the ranked lists. In figure 3 we report statistics under different configurations, i.e., baseline (Bas), bias-reducing (R), bias-enhancing (E) on datasets CUHK03 and Market-1501 for pose (top) and camera bias (bottom). The left and middle figures show the probability that a negative and positive sample, respectively, with the same bias appears at a certain rank position. We also report $nauc_{10}$ aggregating probability values over the first $10$ rank positions. The bias is clearly visible for the baseline (black curve) significantly higher than the ideal case of no bias (grey dotted curve). Bias-reducing branch (R) leads to a drop in $nauc(10)$ by $2\%$ from $0.57$ to $0.55$ for pose and from $0.57$ to $0.55$ for camera bias, which further shows its effectiveness. In contrast, the bias-enhancing branch increases the probability of biased samples occurring higher in the rank, e.g., increasing values of $nauc(10)$.

Similarly, we analyze the behaviour of bias in positives. Although they are not included in the evaluation protocol, we perform this experiment to confirm that the proposed approach does not penalize them, which would have a negative effect in a practical scenario where all retrieved positive samples matter. This is clearly visible in figures 3 (middle) where the probability curves and $nauc(10)$ values remains nearly the same for baseline as well as for R and E branches.

Finally, in figure 3 (right) we show that the influence of bias on the rankings can be controlled by varying the parameters $\lambda_{Db}$ in the bias-reducing and enhancing branches. For example, increasing $\lambda_{Db}$ from $5\cdot 10^{-3}$ to $5\cdot 10^{-2}$, leads the bias-reducing branch (blue curves) to decrease the $nauc(10)$ by $0.5$ in same pose probability (top) and by $0.2$ in same camera probability (bottom). On the other hand, decreasing $\lambda_{Db}$ from $5\cdot 10^{-2}$ to $5\cdot 10^{-3}$ (red curves) in bias-enhancing boosts the probability to retrieve a biased sample from $0.55$ to $0.58$ for the pose and from $0.22$ to $0.24$ for the camera.

4.4 Comparative evaluation

We compare our approach against the state of art methods, in particular GAN, attention, background bias, and other adversarial learning techniques in tables 2, 3 and 4 for Market-1501 and Duke, CUHK03 as well as CropCUHK03, respectively. Our approach is referred to as R+E (bias reducing+enhancing) followed by bias type. In order to make it more complete we also indicate for each method if external data (D+), random erasing augmentation strategy (A+) and re-ranking (RR+) are adopted, as this practices positively affect the re-ID performance.

Market-1501 and DukeMTCM-reID. In table 2 we observe that our proposed method that learns to reduce the pose and camera bias R+E pose+cam improves over the existing GAN based methods, with Rank1 of $90.4\%$ on MARKET-1501 and $82.3\%$ on DUKEMTCM-reID. Compared to the other adversarial learning solutions, we also outperform DR-GAN [27] by a large margin, i.e. by $3.7\%$ in MARKET-1501 and by $5.3\%$ in DUKE. Note that DR-GAN [27] was originally designed for disentangling representations in face identification task and this results reveals the limitation of the method to generalize for articulated bodies. We also compare to FD-GAN [25], which addresses the pose bias. Our method shows comparable results in MARKET-1501 improving slightly the mAP from $77.7\%$ to $77.9\%$, and giving Rank1 score of $82.2\%$ in DukeMTCM, which is higher by $2.2\%$ than FD-GAN. In addition, we outperform MaskG [10] by $6.6\%$, which also tries to address the background bias. These results show the advantage of our method which can deal with different types of bias and improves upon methods targeting specific bias. Moreover, our method performs as well as the attention based networks, that adopt external training data and random erasing augmentation technique. Finally, with re-ranking incorporated R+E (RR+) pose + cam our method with Rank1 $93.1\%$ in Market-1501 and $85.2\%$ in Duke, outperforms most of the methods in this comparative evaluation.

CUHK03 with new $700/767$ protocol is used to obtain results in table 3 for both, labeled (L) and detected (D) splits. Our framework (R+E pose) shows very competitive scores and outperforms MaskG [10] and Pose-transfer [2], which address background and pose bias, respectively. Incorporating re-ranking R+E pose (RR+) leads to Rank1 of $75.5\%$ in labeled and $73.4\%$ in detected splits, outperforming by a large margin all the compared approaches, except $ACCN(D+)$, which uses external data for training.

CropCUHK03. Finally, table 4 shows results for our method in partial re-ID problem on CropCUHK03 and compares it against other recent techniques. In particular, Partial Match Net [15] was specifically designed to handle partial views by additional networks dealing with alignment and hallucination of missing parts. Our method (R+E body part + pose) significantly outperforms Partial Match Net in Rank1 from $31.3\%$ to $43.5\%$, improving the state of art result on this dataset. The results are reported for the same evaluation settings. Moreover, incorporating re-ranking further boosts Rank1 performance to $54.3\%$.

5 Conclusion

We proposed an approach to person re-ID based on adversarial training to extract person representation focused on ID specific cues while allowing to control the influence of bias that many re-ID benchmarks suffer from. Via bias-reducing and bias-enhancing branches we can optimize the balance of features related to re-ID and to the bias. The main novelty is in the training strategy that can be easily incorporated in other recognition problems. We have analyzed the impact of various type of bias in re-ID task, i.e., pose, body part, camera view presenting a methodology applicable to different bias factors. Furthermore, we demonstrate that leveraging two opposite training processes (bias-reducing and enhancing) which either suppress or emphasize bias related features, gives more robust ID representations due to the complementary content of the features. We carried out an extensive evaluation on four different benchmarks in comparison to a number of recently proposed methods. The proposed approach consistently improves the re-ID performance compared to other state of the art techniques.