OmniDRL: Robust Pedestrian Detection using Deep Reinforcement Learning on Omnidirectional Cameras

TL;DR

This paper introduces OmniDRL, a deep reinforcement learning approach that effectively detects pedestrians in omnidirectional images by leveraging image distortion, providing accurate world-position detection without pre-processing, and outperforming existing methods.

Contribution

The paper presents a novel deep reinforcement learning technique that directly utilizes omnidirectional image distortion for robust pedestrian detection, eliminating the need for pre-processing.

Findings

Outperforms state-of-the-art methods in omnidirectional pedestrian detection

Provides accurate 3D world-position of pedestrians

Avoids computationally expensive pre-processing steps

Abstract

Pedestrian detection is one of the most explored topics in computer vision and robotics. The use of deep learning methods allowed the development of new and highly competitive algorithms. Deep Reinforcement Learning has proved to be within the state-of-the-art in terms of both detection in perspective cameras and robotics applications. However, for detection in omnidirectional cameras, the literature is still scarce, mostly because of their high levels of distortion. This paper presents a novel and efficient technique for robust pedestrian detection in omnidirectional images. The proposed method uses deep Reinforcement Learning that takes advantage of the distortion in the image. By considering the 3D bounding boxes and their distorted projections into the image, our method is able to provide the pedestrian's position in the world, in contrast to the image positions provided by most state-of-the-art methods for perspective cameras. Our method avoids the need of pre-processing steps to remove the distortion, which is computationally expensive. Beyond the novel solution, our method compares favorably with the state-of-the-art methodologies that do not consider the underlying distortion for the detection task.

Tables

Table 1. (a) Update Position

Action	Position
$a_t={\rho }^{+}$, $a_t={\rho }^{-}$	${[\rho +a_t,\beta ,z]}^T$
$a_t={\beta }^{+}$, $a_t={\beta }^{-}$	${[\rho ,\beta +a_t,z]}^T$

Table 2. (a) Update Position

Action	Position
$a_t={\rho }^{+}$, $a_t={\rho }^{-}$	${[\rho +a_t,\beta ,z]}^T$
$a_t={\beta }^{+}$, $a_t={\beta }^{-}$	${[\rho ,\beta +a_t,z]}^T$

Table 3. (b) Update Dimension

Action	Dimension
$a_t=w^{+}$, $a_t=w^{-}$	${[w+a_t,h]}^T$
$a_t=h^{+}$, $a_t=h^{-}$	${[w,h+a_t]}^T$

Table 4. TABLE II: This table shows the position errors of ρ 𝜌 \rho (m) and β 𝛽 \beta (rad) for “DQN” and “Multi-task” approaches. The difference of the position obtained when triggered and the ground truth is computed, and then the Root Mean Square Error (RMSE) and the error Standard Deviation (Std).

	RMSE $\rho$	RMSE $\beta$	Error Std $\rho$	Error Std $\beta$
DQN	$0.6748$	$0.0875$	$0.4097$	$0.0837$
Multi-Task	$0.5541$	$0.0316$	$0.5506$	$0.0318$

Table 5. TABLE III: This table shows a comparison between the “DQN” and “Multi-task” methods in terms of the average steps needed to correctly trigger the detection and the average IoU, which evaluates the detection regarding the distorted ground truth. The correct percentage of detection and the raining step ( i.e. , the actions) are also shown.

	Average Steps	Average IoU	Correct (%)	Training Steps
DQN	$27.85$	$0.623$	$71.3$	$1.306\times {10}^6$
Multi-task	$21.39$	$0.718$	$86.5$	$0.936\times {\mathrm{𝟏𝟎}}^{\mathrm{𝟔}}$

Table 6. TABLE IV: This table shows the average steps in the test sequence to set the trigger correctly for the different methods, the average IoU, that evaluate the distorted image on the distorted ground truth, the correct percentage of detections on the test dataset, and the final training steps. The average IoU is only computed for those methods, because the projection of the bounding box to the omnidirectional image after its undistortion in “SotAU” and “FasterU” cannot be obtained.

	Average Steps	Average IoU	Correct (%)	Training Steps
SotA[10]	$62.65$	$0.385$	$83.9$	$0.975\times {10}^6$
SotAU[10]	$28.464$	–	$77.2$	$1.385\times {10}^6$
Faster[50]	–	$0.608$	$28.6$	–
FasterU[50]	–	–	$68.6$	–
Ours	$21.39$	$0.718$	$86.5$	$0.936\times {\mathrm{𝟏𝟎}}^{\mathrm{𝟔}}$

Equations

$${\mathbf{n}}_p^{+}=\mathcal{H}\left(\mathbf{x},\xi \right)=\left(\frac{x_1}{r},\frac{x_2}{r},\frac{x_3+\xi }{r}\right),$$

$$\tilde{\mathbf{i}}={\tilde{\mathbf{i}}}^{+}=\mathcal{F}\left(\mathbf{x},\xi \right)=\left(\frac{n_{p,x}^{+}}{n_{p,z}^{+}},\frac{n_{p,y}^{+}}{n_{p,z}^{+}}\right).$$

$$\mathbf{i}=\left[\begin{array}{cc}{f}_1\eta & f_1\eta \alpha \\ 0& f_2\eta \end{array}\right]\tilde{\mathbf{i}}+\left[\begin{array}{c}{u}_0\\ v_0\end{array}\right],$$

$$\mathbf{C}=\left[\begin{array}{ccc}{l}_1^2\left(1-{\xi }^2\right)-l_3^2{\xi }^2& l_1{l}_2\left(1-{\xi }^2\right)& l_1{l}_3\\ l_1{l}_2\left(1-{\xi }^2\right)& l_2^2\left(1-{\xi }^2\right)-l_3^2{\eta }^2& l_2{l}_3\\ l_1{l}_2& l_2{l}_3& l_3^2\end{array}\right],$$

$$\mathcal{G}(\mathbf{x}_{1},\mathbf{x}_{2})=\big{\{}(\widetilde{i}_{x},\widetilde{i}_{y})\in\mathbb{R}^{2}\ :[\widetilde{i}_{x}\ \widetilde{i}_{y}\ 1]^{T}\mathbf{C}[\widetilde{i}_{x}\ \widetilde{i}_{y}\ 1]=0\\ \wedge\ \widetilde{i}_{1,x}<\widetilde{i}_{x}<\widetilde{i}_{2,x}\ \wedge\ \widetilde{i}_{1,y}<\widetilde{i}_{y}<\widetilde{i}_{2,y}\ \big{\}},$$

$$a_t\in \mathcal{A}=\{{\rho }^{+},{\rho }^{-},{\beta }^{+},{\beta }^{-},w^{+},w^{-},h^{+},h^{-},\sigma \},$$

$$\begin{gathered} {\mathbf{s}}_t=\mathcal{I}\left(\mathcal{G}({\mathbf{x}}_i,{\mathbf{x}}_j)\right)\in \mathbf{S}, \\ \text{with}(i,j)\in \{(1,2),(1,3),(2,4),(3,4)\}, \end{gathered}$$

$$\begin{gathered} L_{drl}({\mathbf{\theta }}_t)= \\ {\mathbb{E}}_{(s_t,a_t,r_t,s_{t+1})\sim U({\mathbf{D}}_t)}\left[(q_j^{DDQN}-Q(s_t,a_t;{\mathbf{\theta }}_t){)}^2\right], \end{gathered}$$

$$q_j^{DDQN}=r_t+\gamma Q(s_{t+1},\arg \underset{a_{t+1}}{\max }Q(s_{t+1},a_{t+1};{\mathbf{\theta }}_t);{\mathbf{\theta }}_t^{-}),$$

$$a_t^{\star }=\arg \underset{a_t}{\max }Q(s_t,a_t;{\mathbf{\theta }}^{\star }).$$

$$\begin{gathered} L_{cls}({\mathbf{\theta }}_t)= \\ {\mathbb{E}}_{\sim U({\mathbf{D}}_t)}\left[-y_c\log (p({\hat{y}}_1;{\mathbf{\theta }}_t))-(1-y_c)\log (p({\hat{y}}_0;{\mathbf{\theta }}_t))\right], \end{gathered}$$

$$y_c^{\star }=\arg \underset{i}{\max }p({\hat{y}}_i;{\mathbf{\theta }}^{\star }),$$

Full text

I Introduction

Object detection has been one of the most relevant research topics in the last two decades, in both robotics and computer vision communities. Along with the availability of larger image datasets (e.g., [1, 2, 3, 4]) and the increase of hardware capabilities, based on GPUs, we have observed a significant improvement in detection and classification [5]. Nowadays, pedestrian analysis is mainly addressed using Deep Learning (DL) techniques (e.g., [6, 7, 8]) which attempt to learn high level abstractions of the data. One of the most important and well-known techniques in DL are the Convolutional Neural Networks (CNNs) [6], which halved the error rate for image classification. Very recently, there have been some alternative methods, namely the use of deep Reinforcement Learning (RL), where a policy is created in order to maximize the rewards of an agent [9, 10]. On the other hand, omnidirectional cameras have been used in an increasing number of applications ranging from surveillance to robot navigation [11, 12, 13, 14], which require a wide Field of View (FoV) of the environment. This wide FoV is usually obtained by means of distortion [15].

An alternative solution for Pedestrian Detection (PD) using omnidirectional cameras would be to first apply undistortion techniques [16, 17, 18], followed by conventional detection tools. However, these undistortion procedures suffer from the following shortcomings:

1. They are too computationally expensive;
2. They generate images whose pedestrians cannot fit properly in regular bounding boxes111Although undistorted images keep the perspective projection constraints (i.e., straight lines in the world will be projected into straight lines in the image), the objects will be stretched. This means that the objects should not be approximated by regular bounding boxes, which are used in most of the DL techniques for object detection (we note that there are alternatives that do not consider regular bounding boxes in [19]).; and
3. They introduce artifacts in the undistorted image that affects the detection accuracy.

Fig. 1 shows the following steps that describe our approach: (i) an omnidirectional image containing the pedestrian (“Input Image”); (ii) our parametrization, using cylindrical coordinates that best represents the 3D environment for omnidirectional cameras (“Environment Setup”); and (iii) the 2D representation of the distorted bounding box states given by the attention mechanism, performed through the agent’s actions, for the estimation of the pedestrian 3D position (“Action results”).

I-A Related Work

This section revises related approaches in object/pedestrian detection and omnidirectional vision systems.

Object Detection and Classification: Object detection has a wide range of applications comprising quite diverse research areas, such as artificial intelligence, video surveillance and multimedia systems. This problem has seen a significant progress in the last decade. Looking at the performance of algorithms in, e.g., PASCAL VOC [3] datasets, we can easily conclude that the progress slowed from 2010 onward. However, the use of CNNs [6] has led to a significant improvement in the accuracy of image classification, for the Imagenet Large Scale Visual Recognition Challenge (ILSVRC) [20]. Soon, more methodologies became available, such as: SppNet [21], R-CNN [22, 23], Mask [24], Fast [25], Faster R-CNN [26], SSD [27] and YOLOv3 [28].

Although the methods presented above are valuable contributions in the field, we follow a different approach based on deep RL. One of the reasons behind is that deep RL can significantly reduce the inference time for object detection, when compared to an exhaustive search [10, 29]. In [10] it is proposed the use of a Deep Q-Network (DQN) [9] for object detection, enabling the use of a small amount of training data without compromising the accuracy. The Q-Network has also been used in other domains, such as medical imaging analysis, e.g., [30, 29]. However, the above frameworks have not been explored to model images containing objects/pedestrians with high distortion, such as the ones encountered in omnidirectional systems.

Since the classification task is highly dependent on the bounding box proposal, we propose a multi-task network that learns both tasks at the same time (see Fig. 2). It will be experimentally shown that the proposed methodology reduces the computational time and increases classification and detection accuracy. In our framework a hard222The term hard means that the first convolution layers are shared, and then a branch is created for each sub-network, see Fig. 2. parameter sharing network [31] is used. In the multi-task methodology, each sub-network is trained alternatively.

Omnidirectional Vision: Omnidirectional cameras achieve a wide FoV from two distinct camera configurations: 1) Using special types of lenses (dioptric cameras), such as fish-eye [32]; and 2) Combining quadric mirrors with perspective cameras (catadioptric cameras) [33, 34, 35]. Image formation for these systems has been largely studied in the literature. [34] describes the necessary conditions to ensure that a catadioptric system is a central camera. In general, these are non-central [15]. An alternative way to build an omnidirectional camera would be a setup with multiple perspective cameras. However, this requires finding correspondences between features and merging images from different cameras. This requires a lot of computational effort, while merging images from different views, details and properties of the environment are lost [36].

A small amount of research addressed the object/person detection in omnidirectional images. One work that is related to our approach is presented in [37]. The authors adopt a conventional camera approach that uses sliding windows and Histogram of Gradients (HOG) features. Since the shape of the sliding window depends on the position of the person w.r.t. the camera, the HOG filters are trained with perspective cameras and then changed to account for the distortion. Although this method does not require the image unwrapping, avoiding expensive computational resources, it suffers from artifacts caused by changes in the resolution. By introducing the deep RL in omnidirectional images (OmniDRL), we aim at avoiding the above shortcomings.

There have been some advances in feature extraction for omnidirectional systems (e.g., [38]). However, this is still a difficult task to be accomplished due to the distortion in the image. The shape of the features depend on their position on the image (due to the distortion). This is why the traditional feature extraction methods are not suited for these systems. Some methods exist for soccer robots using catadioptric systems [11]. However, they require the knowledge of the object shape and color. There are also tracking methods for omnidirectional cameras based on object’s motion, by using background subtraction [39], and its ego-motion [40].

Other works address this problem by first applying transformations to the images, followed by the use of conventional techniques, e.g., [41]. In [42], the authors search for objects directly in omnidirectional images, but do not consider the underlying distortion. In [43], the robustness of affine co-variant features as a function of the distortion is analyzed.

I-B Contributions

We revisit the deep RL in the new context of pedestrian detection in highly distorted scenarios, namely using omnidirectional cameras. The problem formulated herein considers solutions that implicitly take into account distortion’s effects on the detection. Our main contributions are as follows:

Introduction of a novel PD in omnidirectional cameras, inspired in the Deep Q-Network (DQN) methodology and classification tasks. The proposed approach comprises an artificial agent (i.e., DQN agent) that can automatically learn policies directly from high dimensional data, to perform actions in the environment that maximizes the cumulative reward. 2. 2.

Proposal of a novel multi-task learning strategy that contains two sub-networks: (i) a DQN, based on deep Reinforcement Learning (see top branch in Fig. 2); and (ii) a second classification network which provides a prior for the pedestrian localization that helps the job of the DQN sub-network (see bottom branch in Fig. 2). 3. 3.

Opposing to the related methods, where the actions are performed in the image domain, our approach is able to perform actions in the world coordinate system. This allows us to know the location of the pedestrian in the 3D environment, i.e., the pedestrian’s position relative to the camera system can be obtained. This is a useful feature in human-awareness navigation [8, 44].

II Line Segment Projection using Omnidirectional Cameras

We start by defining the image formation model, Sec. II-A. Then, we describe the projection of 3D line segments, required to define the bounding box projection (Sec. II-B).

II-A Image Formation

To deal with general omnidirectional cameras, it is used the spherical model [16, 17, 18]333Notice that, this model can also be used to represent fisheye cameras [45, 18].. Let us assume a unit sphere centered at the origin of the mirror’s reference frame. A 3D point $\mathbf{x}=(x_{1},x_{2},x_{3})\in\mathbb{R}^{3}$ is projected onto the sphere’s surface (point $\mathbf{n}\in\mathcal{S}^{2}\subset\mathbb{R}^{3}$), resulting in a pair of antipodal points $\left\{\mathbf{n}^{+},\mathbf{n}^{-}\right\}$ (see Fig. 3LABEL:sub@fig:sphereProj_points for more details). The antipodal point closer to $\mathbf{x}$ (i.e., $\mathbf{n}^{+}$ as the convention in the figure) is chosen to be the point projected to the image.

Afterwards, the reference frame is changed and it will be centered at $\mathbf{c}_{p}=(0,0,\xi)$. In the new reference frame, the point $\mathbf{n}^{+}$ will be given by the function $\mathcal{H}:\mathbb{R}^{3}\rightarrow\mathbb{R}^{3}$:

[TABLE]

where $r=\sqrt{x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}$. Then, $\mathbf{n}^{+}$ will be projected to the normalized image plane $z=1$ mapped by the function $\mathcal{F}:\mathbb{R}^{3}\rightarrow\mathbb{R}^{2}$:

[TABLE]

Finally, the resulting point has to be mapped on the image plane through the camera projection:

[TABLE]

where $f_{1}$ and $f_{2}$ are the focal lengths, $u_{0}$ and $v_{0}$ are the coordinates of the principal point in the image, and $\alpha$ is the skew parameter [46]. Finally, $\eta$ is the distance from the center of the first reference frame $\mathcal{O}$ to the image plane. From (3), one can see that the projection of a point to the image plane is a function of the parameters $\xi$ and $\eta$, which depend on the omnidirectional camera parameters.

II-B 3D Line Segment Projection

In central omnidirectional cameras, 3D straight lines are parameterized by a plane $\bm{\Pi}$ (interpretation plane [46]), which comprises by the center of the projection sphere ($\mathcal{O}$ in Fig. 3LABEL:sub@fig:sphereProj_lines) and the respective 3D straight line (notice that, without lost of generality, all lines on this plane will be equally parameterized). Let us denote a line as $\mathbf{l}=(l_{1},l_{2},l_{3})\in\mathbb{R}^{3}$.

By definition and considering the above parameterization, for two 3D points on the line (say $\mathbf{x}_{1}$ and $\mathbf{x}_{2}$), one can represent the line using $\mathbf{l}=\mathbf{x}_{1}\times\mathbf{x}_{2}\in\mathbb{R}^{3}$ (line’s moment which is perpendicular to $\bm{\Pi}$), and every point $\mathbf{x}$ belonging to the line must verify $\mathbf{l}^{T}\mathbf{x}=0$.

Thus, following [17], the image of a 3D straight line on the normalized plane must also belong to a quadric that is defined by the intersection of the plane $\bm{\Pi}$ and the unit sphere. See Fig. 3LABEL:sub@fig:sphereProj_lines for more details. After some algebraic manipulations, one can obtain a quadric equation $\mathbf{x}^{T}\mathbf{C}\mathbf{x}=0$, where:

[TABLE]

represents the curve in the sphere. To get a representation of the curve in the image plane, we have to consider the constraint $z=1$, i.e., the projection to the normalized plane. To conclude, one can define the projection curve (image of the line’s segment) into the plane by:

[TABLE]

where $(\widetilde{i}_{1,{x}},\widetilde{i}_{1,{y}}\,)=\mathcal{F}({\bf x}_{1},\xi)$ and $(\widetilde{i}_{2,{x}},\widetilde{i}_{2,{y}})=\mathcal{F}({\bf x}_{2},\xi)$ (see (2) and also Fig. 3LABEL:sub@fig:sphereProj_lines). Finally, to the each point that belongs to the curve, a final mapping must be applied to project them into the image plane, which is obtained by applying (3).

III Robust Detection

In this section, we present the principles behind the training and inference of the deep RL agent and, the classification network used. We define the agent’s training and how it infers its actions in Sec. III-A. Further, in Sec. III-B, the same process is done for the classification task using CNNs.

III-A Deep Reinforcement Learning

This section describes how object detection is performed using deep RL. Recently, [10] have proposed the use of a Q-network as a way to improve the efficiency on the detection of an object. The main challenge is to the adapt our DQN for object detection through movements applied to a bounding box, in order to deal with the high distortions created by the omnidirectional imaging device. In order to have a 3D position of the pedestrian, it was considered a 3D bounding box representation in a 3D environment (Fig. 1 - “Environment Setup”).

The DQN is a deep Q-Network parameterized by ${\bm{\theta}_{t}}$ that, for a given state $s_{t}$ approximates the Action-Value Function $Q(s_{t},a_{t};{\bm{\theta}}_{t})$ for a set of nine actions, that are function of the angle ($\beta$), distance ($\rho$), height ($h$) and width ($w$):

[TABLE]

The effects of each action $a_{t}$ on the bounding box position and dimension is shown in Tab. I (see “Environment Setup” in Fig. 1). The Q-values given by $Q(s_{t},a_{t};{\bm{\theta}}_{t})$ represent the maximum discounted future reward of the Bellman equation [47]. Based on the 3D line’s projection defined in the previous section, each state (bounding box projection (5)) can be formally defined by:

[TABLE]

where $\mathbf{x}_{i}$ for $i=\{1,2,3,4\}$ are the 3D bounding box corners, and $\mathcal{I}(.)$ represents the resulting image cropped within the bounding box limits.

The training process of the DQN follows a deep Q-learning algorithm proposed by Mnih [9], where the authors used two main key features. First, the process makes use of a memory $\mathbf{D}_{t}=\{e_{1},...,e_{t}\}$ that is built with the agent’s experiences $\mathbf{e}_{t}=(s_{t},a_{t},r_{t},s_{t+1})$. These samples are drawn uniformly from the memory and will be used as a batch to train the prediction network. Second, a target network containing the parameters $\bm{\theta}_{t}^{-}$ computes the target values that allows the DQN updates. These values $\bm{\theta}_{t}^{-}$ are held unchanged and updated periodically.

One shortcoming of the DQN is that the action selection and the corresponding evaluation tend to use the same Q-values, leading to overoptimistic estimates. To tackle the above limitation we use the Double DQN (DDQN - [48]). Therefore, the loss function that models $Q(s_{t},a_{t};\bm{\theta}_{t})$ minimizes the following mean square error on the modified version of the Bellman equation:

[TABLE]

where:

[TABLE]

where $U(\mathbf{D}_{t})$ represents the batch retrieved from the memory $\mathbf{D}_{t}$ with uniform distribution $U$.

In the training process, the agent must first explore the environment in order to better understand which actions will maximise its future reward and then exploit those states that leads it to that objective. We choose a strategy that will balance the two phases i.e., exploration and exploitation. For our training, we use the Softmax action selection method using a Gibbs or Boltzmann distribution [49].

For inference, the trained DQN model is parameterized by $\bm{\theta}^{\star}$ learned in (8) that outputs the action-value function for the current state $s_{t}$. Formally, the action to follow from the current state is defined by:

[TABLE]

III-B Classification Branch

In this section, we describe how the classification network is trained (see classification branch in Fig. 2). The main goal in a classification task is to assign a class to an object, within a proposed region. These regions can be represented as: rectangular bounding box [26], segmented region [24] or, as we propose, distorted bounding box.

The purpose is to classify the proposed region as having the pedestrian or not, thus we consider only two classes $y_{c}=\{0,1\}$ (No Pedestrian and Pedestrian, respectively). Assuming one pedestrian per image, we consider that for each image $\mathcal{I}(.)$ a label is assigned. Then, our primary goal is to create a network, described by the parameters ${\bm{\theta}}_{t}$, capable of generating a prediction of the desired class for a hold out test image. Therefore, it was used a CNN to model this behavior by minimizing the classification error of a batch of labeled images, according to cross-entropy’s mean:

[TABLE]

where $y_{c}$ is the labeled class; $p_{i}(\widehat{y}_{i};{\bm{\theta}}_{t})$ is the computed probability for each class; ${\bm{\theta}}_{t}$ are the network’s weights and $\widehat{y}_{i},\ i\in\{0,1\}$ are its estimates (for each class at iteration $t$); and $U(\mathbf{D}_{t})$ is the uniformly retrieved batch of labeled images from the memory used in the deep RL system. This probability is computed using the Softmax function.

For inference, it is chosen $i$ that has the highest $p_{i}(\widehat{y}_{i};{\bm{\theta}}_{t})$:

[TABLE]

where ${\bm{\theta}}^{\star}$ are the optimal parameters of the model learned in (11). The obtained classification will provide the initial state position for the deep RL branch (see Sec. III-A), in testing.

IV Experimental Results

This section is organized as follows. First, in Sec. IV-A we compare the performance between the DQN (see top branch in Fig. 2) and the advantages by coupling a classification network (bottom branch in Fig. 2) that constitutes our multi-task learning methodology. The second part of the experiments (Sec. IV-B) concerns a comparison of the proposed multi-task learning method with the State-of-the-Art: the object detection method proposed by Caicedo [10]; and the ”Faster R-CNN” using a ResNet-101 architecture [50], pre-trained with the PASCAL VOC 2007 and 2012 datasets.

As far as we know, there are no available datasets in omnidirectional settings for PD, as well as the corresponding labels. Thus, we acquired a new one using three different environments in our laboratory with several subjects. With a total of 921 images, $70\%$ of them are used for training and $30\%$ are used for testing. These images were obtained with [51]’s Flea3 camera attached to a hyperbolic mirror, modeled and calibrated by the central catadioptric camera system [18]. Our method was developed using TensorFlow and OpenCV. The dataset are available on the authors’ webpage.

IV-A Comparison between Deep RL and Multi task learning

The architecture of the DQN network comprises five convolutional layers and two fully-connected layers, where the input is a $224\times 224$ image size, and the output contains the Q-values for the nine possible actions (see top network in Fig. 2). The multi-task network contains a second classification network, having three shared convolutional layers, two more convolutional layers and two fully-connected layers for each branch. Now, in addition to the nine Q-values, the Softmax probability is retrieved for the two classes (presence or absence of the pedestrian), in the classification network. For the training parameter we considered $\tau=0.6$, $\alpha=10$, $C=15000$ steps and a fixed learning rate of $0.0001$.

For the purpose of test initialization, we choose a fixed $\rho$ closed to the camera center and a fixed dimension $\mathbf{dim}_{0}$ with a relatively large size. Afterwards, six values for the $\beta$ parameter are randomly selected in order to cover the whole image domain, aiming to detect some pedestrian location. For the other values, no detection has been triggered (a maximum number of 100 steps has been used).

For evaluation, a hold-out test sequence is used to compute the average steps required to detect the pedestrian, as well as the average of the Intersection over Union (IoU) over the final detections. Notice that the steps correspond to actions taken and are counted from the first initial position until the detection of the pedestrian has been triggered.

Tab. II shows the obtained errors for the duplet $(\rho,\beta)$ for the two approaches, DQN and Multi-task (DQM & classification branch). The proposed solutions exhibit small errors in the detection task in spite of the environment’s complexity. However, by observing the position error $\rho$ and its standard deviation, the errors obtained are larger than it would be expected. This is justified by the IoU not fully demonstrate the changes on the distortion given by $\rho$, i.e., $\mathbf{s}_{gt}$ and $\mathbf{s}_{t+1}$ can be identical ($IoU(\mathbf{s}_{gt},\mathbf{s}_{t+1})>0.65$), even though the current state is further away from the ground truth, due to the distortion in central catadioptric systems being radial. Nonetheless, as we can also examine in Tab. II, the position error of $\beta$ is small, thus small changes in $\beta$ will lower the IoU, for the same reason as before. The distortion in this kind of system will have a higher effect when the bounding box moves along the radius.

Table III shows the superiority of the Multi-task approach. This is achieved in both accuracy (better IoU scores) and efficiency (less actions needed for the detection). From the above, we can state that the classification branch, helps on the training and allow us to get an initial estimation for testing. In the next section, we perform the comparison with other related methodologies, and the proposed multi-task network will be termed as “Ours”.

IV-B Comparison with the State-of-the-Art Methods

In this section, it is performed a comparison between the proposed multi-task framework (“Ours”) within the environment discussed in Sec. III, against two state-of-the-art approaches using rectangular bounding boxes: 1) Caicedo’s method; and 2) a pre-trained Faster R-CNN. For the state-of-the art methods, two distinct settings are used: i) distorted images acquired by our omnidirectional imaging device (termed as “SotA” and “Faster”, respectively); and ii) unwrapped images obtained from the original images (in the same way, termed as “SotAU” and “FasterU”). Note that the environment where the actions are performed for these methods are in the image domain while ours is in 3D.

The network “Ours”, “SotA” and “SotAU” have five convolutional layers and two fully-connected layers, where the input is a $224\times 224$ image size, and the output contains the Q-values for the nine possible actions. Notice that we can maintain the same dimension of actions as in [10], thus we can keep the same architecture to train both methods. “Ours” has two more convolutional layers and two fully-connected layers for each branch (see Fig. 2). In addition to the nine Q-values output, in the “Ours” method, the Softmax probability is also retrieved for the two classes in the classification network. The training parameters are the same for all the above networks. These three networks are trained from scratch. It is important to mention that we used the Faster R-CNN as a baseline approach, in order to verify how a pre-trained network for perspective images behaves when distorted and unwrapped images are tested. Thus, average and training steps will not be evaluated for this approach.

For evaluation purposes, we used the test dataset to compute the average steps required to detect the pedestrian, as well as the average IoU for the final detections. Notice that the steps correspond to actions taken and are counted from the first initial position until the detection of the pedestrian has been triggered. For the IoU computation, we only take in consideration the three methods that deal with the distorted image (i.e., “SotA”, “Faster” and “Ours”). From Tab. IV, it can be seen that our method takes much less steps to achieve the correct detection than the “SotA”, even though the environment is much more complex, as the coordinate system is set in the world coordinates and not in the image domain. An example illustrating the steps required for each algorithm can be seen in the Fig. 4. Regarding the perspective method on the undistorted image (“SotAU”), we obtained a small number of average steps due to the simplicity of its environment, since the image is much smaller and the initialization of the initial bounding box already covers a big part of the environment. Also note that our method reaches an IoU improvement of over $33.3\%$ when comparing to the “SotA” method and over $11\%$ regarding the “Faster” method. However, as it can be seen, the correct detection percentage (shown in Tab. IV) for the “Faster” is less $57.9\%$ when compared to our algorithm. One can assume that the high percentage on the average IoU, for the “Faster” method, is related with images where the pedestrian is located farther from the imaging device, where the distortion’s effects are mostly neglected.

In the experiments, we consider a correct detection when the trigger is signaled above a threshold444The same value of $\tau$ is used in training. or, in the case of the “Faster” and “FasterU”, when a pedestrian is detected in the image. From the Tab. IV we can see that the proposed method “Ours” triggers the detection using a smaller number of actions and deals better with the distortion when compared to both “SotA” and “SotAU”. Our approach outperforms $2.5\%$ points over the previous solutions. In addition, one can state that the “SotAU” method outperforms the “FasterU” on the correct detection percentage, since the “SotAU” method was trained on images that contain artifacts caused by the unwrapping process while the “FasterU” was already pre-trained with images acquired from perspective cameras.

V Discussion

In this work, we have presented a novel methodology for PD in omnidirectional systems. The approach uses the underlying geometry of general central omnidirectional cameras along with a multi-task learning methodology. From the extensive conducted evaluation, the methodology is able to accurately detect pedestrians without resorting to undistortion procedures which are computationally expensive. Moreover, our proposal can provide the 3D world position of the pedestrian instead of 2D image coordinates. Also, it needs a smaller number of agent actions to reach the correct detection, and exhibits a higher accuracy on the final bounding box representation. This is due to the fact that our framework inherently holds high levels of distortions where the bounding box is represented in cylindrical coordinates which are tailored for these omnidirectional environments.