Constrained Design of Deep Iris Networks

TL;DR

This paper introduces a constrained optimization approach to designing deep iris networks, achieving state-of-the-art performance with reduced model size and computational cost, while also analyzing the optimality of classic IrisCode methods.

Contribution

It proposes a novel constrained optimization framework for automated iris network design that balances accuracy, model compactness, and computational efficiency.

Findings

Achieves state-of-the-art iris recognition accuracy.

Reduces model size and FLOPs compared to existing networks.

Provides insights into the optimality of classic IrisCode methods.

Abstract

Despite the promise of recent deep neural networks in the iris recognition setting, there are vital properties of the classic IrisCode which are almost unable to be achieved with current deep iris networks: the compactness of model and the small number of computing operations (FLOPs). This paper re-models the iris network design process as a constrained optimization problem which takes model size and computation into account as learning criteria. On one hand, this allows us to fully automate the network design process to search for the best iris network confined to the computation and model compactness constraints. On the other hand, it allows us to investigate the optimality of the classic IrisCode and recent iris networks. It also allows us to learn an optimal iris network and demonstrate state-of-the-art performance with less computation and memory requirements.

Tables

Table 1. TABLE I: Statistics of three datasets, ND-CrossSensor-2013, CASIA-Iris-Thousand and UBIRIS.v2 in this research.

	# Subjects	# Images	Distance	Image Resolution	Iris Diameter	Wavelength	Subject Cooperation
ND-CrossSensor-2013	676	111,564	Close-up	640x480	200	NIR	Highly
CASIA-Iris-Thousand	1,000	20,000	Close-up	640x480	180	NIR	Highly
UBIRIS.v2	261	11,102	4-8 meters	800x600	180-80	Visible	Less

Table 2. TABLE II: Case 1 constrained design task.

Inputs	Algorithm hyperparameters	Outputs
K = $K_{IrisCode}$ P = $Inf$	- $𝕆$ = {$3\times 3$, $3\times 5$ $conv$; $2\times 2$ $max$ pool; $2\times 2$ $avg$ pool; $Identity$; $Zero$} - Loss = cross-entropy	- ${\alpha }^{\ast }$ - $w^{\ast }$ - $EER\ast$

Table 3. TABLE III: Computation vs. EER of deep architectures vs. IrisCode. The first row presents the computation ratio between the deep network and IrisCode K R = K d ​ e ​ e ​ p ​ a ​ r ​ c ​ h ​ i ​ t ​ e ​ c ​ t ​ u ​ r ​ e K I ​ r ​ i ​ s ​ C ​ o ​ d ​ e subscript 𝐾 𝑅 subscript 𝐾 𝑑 𝑒 𝑒 𝑝 𝑎 𝑟 𝑐 ℎ 𝑖 𝑡 𝑒 𝑐 𝑡 𝑢 𝑟 𝑒 subscript 𝐾 𝐼 𝑟 𝑖 𝑠 𝐶 𝑜 𝑑 𝑒 K_{R}=\frac{K_{deeparchitecture}}{K_{IrisCode}} ; the second row presents the best E ​ E ​ R 𝐸 𝐸 𝑅 EER achieved by under the selected computation constraints; and the third row presents the accuracy ratio between the deep network and IrisCode E ​ E ​ R R = E ​ E ​ R d ​ e ​ e ​ p ​ a ​ r ​ c ​ h ​ i ​ t ​ e ​ c ​ t ​ u ​ r ​ e E ​ E ​ R I ​ r ​ i ​ s ​ C ​ o ​ d ​ e 𝐸 𝐸 subscript 𝑅 𝑅 𝐸 𝐸 subscript 𝑅 𝑑 𝑒 𝑒 𝑝 𝑎 𝑟 𝑐 ℎ 𝑖 𝑡 𝑒 𝑐 𝑡 𝑢 𝑟 𝑒 𝐸 𝐸 subscript 𝑅 𝐼 𝑟 𝑖 𝑠 𝐶 𝑜 𝑑 𝑒 EER_{R}=\frac{EER_{deeparchitecture}}{EER_{IrisCode}} .

$K_R$	2.0	3.0	4.0	5.0	6.0	7.0	8.0
EER	16.2	15.1	13.3	8.9	5.6	2.9	1.4
$EE{R}_R$	9.3	8.6	7.6	5.1	3.2	1.7	0.8

Table 4. TABLE IV: Case 2 constrained design task.

Inputs	Algorithm hyperparameters	Outputs
K = $K_{ResNet18}$ P = $P_{ResNet18}$	- $𝕆$ = {$3\times 3$, $3\times 5$ $conv$; $2\times 2$ $max$ pool; $2\times 2$ $avg$ pool; $Identity$; $Zero$} - Loss = cross-entropy	- ${\alpha }^{\ast }$ - $w^{\ast }$ - $EER\ast$

Table 5. TABLE V: Case 3 constrained design task.

Inputs	Algorithm hyperparameters	Outputs
K = $K_{FeatNet}$ P = $P_{FeatNet}$	- $𝕆$ = {$3\times 3$, $3\times 5$ $conv$; $3\times 3$, $3\times 5$ $dilated$ conv; $2\times 2$ $max$ pool; $2\times 2$ $avg$ pool; $Identity$; $Zero$} - Loss = pairwise	- ${\alpha }^{\ast }$ - $w^{\ast }$ - $EER\ast$

Table 6. TABLE VI: Comparison with state of the art approaches on ND-CrossSensor-2013 dataset.

	FRR	EER	K	P
IrisCode	3.76%	1.75%	0.5M	5
DeepIris [16]	2.62%	1.31%	20M	192K
FeatNet [32]	1.79%	0.99%	30M	13K
Ours	1.32%	0.68%	19M	11K

Table 7. TABLE VII: Cross-dataset performance of the network discovered.

	CASIA-Iris-1K		UBIRISv2
	FRR	EER	FRR	EER
IrisCode	5.53%	3.46%	14.31%	8.33%
DeepIris [16]	4.25%	2.17%	13.20%	7.12%
FeatNet [32]	3.98%	1.93%	13.93%	6.69%
Ours	3.07%	1.54%	11.12%	5.98%

Equations

$$x_j=\sum _{i<j}{o}^{(i,j)}(x_i),$$

$${\overline{o}}^{(i,j)}(x)=\sum _{o\in \mathbb{O}}\frac{exp({\alpha }_o^{(i,j)})}{Z}o(x),$$

$$\underset{\alpha }{\min }{L}_{val}(w^{\ast }(\alpha ),\alpha )+|\alpha |,$$

$$\#$$

$$w^{\ast }(\alpha )$$

$$max_{r,x_{0},y_{0}}\big{|}G_{\sigma}(r)*\frac{\partial}{\partial r}\oint_{r,x_{0},y_{0}}\frac{I(x,y)}{2\pi r}ds\big{|},$$

$$x(r,\theta )=(1-r)x_p(\theta )+r{x}_s(\theta ),$$

$$y(r,\theta )=(1-r)y_p(\theta )+r{y}_s(\theta ).$$

Full text

1 Introduction

Deep neural networks are extremely effective at automatic feature learning for object recognition by leveraging the power of high capacity models, vast amounts of data and high-end computing infrastructure [27, 1]. Recently, deep networks have shown to be able to automatically learn discriminative features with promising recognition accuracy in the iris recognition setting [19, 11, 32, 16]. However, when compared with handcrafted approaches such as the classic IrisCode [7], it is pertinent to ask a question: do we really need networks that deep with tens or even hundreds of layers and millions of parameters and floating-point operations in the iris recognition setting? Recall that the IrisCode has been very successful with only a few parameters, e.g. orientations and scales of Gabor filters, and it has been very cheap to run (30 ms on a CPU for iris image analysis and creation of an IrisCode) [17].

Despite its superior accuracy, current deep iris networks are unable to achieve two vital properties of the classic IrisCode: the compactness of the model and the small number of computing operations (FLOPs), which are major driving forces for iris recognition to be the favorite choice compared with other biometric modalities. This is due to the inherent large size of the deep networks with millions of parameters and hundreds of layers that are potentially required to achieve the desired accuracy. The benefits of automatic feature engineering come at the cost of significant computational power and memory requirements. Because automatic feature learning is highly desired in the iris recognition context (to both remove the pitfalls in feature design and automatically discover the best feature representation directly from the data), it is urged to understand the trade-off between the superior accuracy by automatic feature engineering and and the excessive computation and network model size we have to adopt for this benefit.

In addition, there are a vast number of possible architectures (i.e. number of layers, filters, connections, etc.) of a neural network, which can be millions in case of deep networks. In the iris recognition setting, the existing architectures can range from several layers (FeatNet [32]) to tens of layers (DeepIrisNet [11], off-the-shelf CNNs [19]) with millions of possible connections between them. Redundant layers, connections and filters would lead to non-compact feature representations and unnecessary computation. This raises two important questions: (i) how can we determine the optimal architecture? and (ii) how do we fully automate the network design process, i.e. automatic feature learning and automatic architecture learning? These are important issues that must be addressed to reach the full potential of deep networks in the iris recognition setting.

One possible solution to the above problems is to leverage recent advances in neural architecture search (NAS) to search for the network architecture that can achieve the best accuracy. Modern neural architecture search relies on reinforcement learning and evolution theory to explore the architecture space to gradually evolve the architecture to better ones. For example, Zoph et al. designed a reinforcement learning agent to explore the architecture search space to find the optimal configurations [33, 34]. Real et al. relied on evolution theory to gradually evolve the network architecture toward higher performing ones [26].

Existing NAS only focuses on improving the accuracy. However, there are other vital factors which are critical to the success in real-world applications. In the iris recognition setting, these include two properties mentioned above: the model compactness and the computation. Both factors have been the main driving forces to the success of modern iris recognition [8, 6, 23]. Taking these factors into account is important especially for practical applications. For example, if the iris recognition system is stored and processed on a high-computing platform, then there needs to be no restrictions on model compactness or matching speed, and the design process can target the best accuracy. In contrast, if the iris recognition system is performed on embedded or mobile systems with limited resources, model compactness and matching speed have to be prioritized.

There are two main objectives of this paper: (1) Develop an algorithm to search for the optimal deep iris network that both satisfies the pre-defined constraints in computation and model compactness and achieves the highest possible performance; and (2) Use this algorithm to investigate the optimality of the classic IrisCode and recent iris networks. To achieve these objectives, we re-visit existing NAS, and re-interpret it with additional constraints to model the design as a constrained optimization problem. Solving this constrained optimization problem allows us to automatically discover the optimal network (the network achieves the highest achievable accuracy within the constraints of the computation and model compactness). In addition, applying this algorithm with two constraints similar to those of the existing approaches, we are able to: (i) understand the optimality of the classic handcrafted IrisCode and recent deep iris networks; and (ii) optimize an existing network architecture to achieve the best accuracy under the same computational and memory cost.

Our major contributions can be summarized as follows:

• •

Our work is the first effort to consider automatic architecture search for deep iris recognition networks.

• •

We re-interpret the search for network architecture and parameters as a constrained optimization with design constraints related to the compactness of the model along with the matching speed.

• •

Our approach results in the full automation of deep iris network design, in both feature learning and architecture learning.

• •

We discover an optimal deep network for iris recognition achieving state-of-the-art performance with less computation and memory required compared to existing deep iris networks.

• •

We provide a way to understand the optimality of the classic IrisCode approach.

The remainder of the paper is organized as follows. Section 2 discusses related feature learning and architecture learning for iris recognition; Section 3 presents how we model the network design as a bi-level constrained optimization problem, Section 4 illustrates our experimental results; and the paper is concluded in Section 5.

2 Related work

This section reviews the existing attempts in the literature on applying deep learning approaches for iris recognition and architecture learning via neural architecture search.

2.1 Feature learning for iris recognition

A number of deep networks have been proposed to take advantage of their automatic feature engineering capability to automatically learn feature representation for iris images, aiming to improve recognition performance of the iris biometric system. Liu et al. proposed a DeepIris network of 9 layers including one pairwise filter layer (similar to a convolutional layer), one convolutional layer, two pooling layers, two normalization layers, two local layers and one fully-connected layer [16]. Gangwar et al. employed more advanced layer types to create two DeepIrisNets for the iris recognition task [11]. The first network, DeepIrisNet-A, contains 8 convolutional layers (each followed by a batch normalization layer), 4 pooling layers, 3 fully connected layers and two drop-out layers. The second network, DeepIrisNet-B, adds two inception layers to increase the modeling capability. To deal with the lack of a large-scale dataset, transfer learning can be used. Nguyen et al. experimented a wide range of off-the-shelf CNNs, which had won the ImageNet challenges (AlexNet, VGG, Inception, ResNet, DenseNet), on the iris recognition task [19]. They showed that these off-the-shelf CNN features can be successfully transferred to the iris recognition task, achieving state-of-the-art recognition accuracy. In contrast, Zhao et al. argued that CNNs may be not optimal for the iris recognition task since iris texture, which is believed to be random, does not exhibit structural information or meaningful hierarchies [32]. Hence they proposed a fully convolutional network named FeatNet without any fully connected layers to retain spatial correspondence with the original input image, leading to higher recognition rates.

All these deep iris networks have their architectures handcrafted, which means potential redundant connections, filters and layers, and no best accuracy guarantee. Learning both a feature representation and an optimal architecture is challenging considering the huge number of potential architectures and their parameters. This challenge will be addressed in this paper.

2.2 Neural Architecture Search

There does not exist any work in network architecture search in the iris recognition setting. In general deep learning, modern architecture search approaches usually rely on Evolution Theory and Reinforcement Learning (RL) to design searching policies. For example, Real et al. brought in the ideas from the natural evolution process to gradually update the architecture of the network to achieve higher accuracies [26, 25]. At each iteration, a number of architectures, called population, are investigated, the best architecture is mutated (i.e. randomly add/remove layers) to generate a new child architecture to be added into the population. Subsequently, the worst architecture [26] or the oldest architecture [25] is discarded to generate a new population. This algorithm is considered as a discrete optimization process. In contrast, Zoph et al. trained a RNN controller to iteratively sample candidate architectures, and trains them to convergence to measure their performance on the desired task. The controller then uses the performance as a guiding signal to find more promising architectures [33, 34]. Parameter sharing can be forced on all child models to improve the search speed at a slight cost of performance [20]. Both evolution and RL approaches are very computationally expensive despite their remarkable performance. For example, obtaining a state-of-the-art architecture for CIFAR-10 required 1800 GPU days of RL [34] or 3150 GPU days of evolution [25].

One of our main motivations arises from a recent body of works on gradient-based architecture search [13, 15] where the problem is encoded as a bi-level optimization problem with one level to optimize based on the architecture and the other level to optimize based on the weights of the chosen architecture. This strategy is able to discover high-performing architectures achieving high classification accuracy in only tens of GPU hours. This strategy is highly desired in the iris recognition scenario to search for the highest-accuracy architecture. However, it is not directly applicable to the iris recognition scenario because in contrast to the natural image recognition scenario, the approaches in the iris recognition scenario need to take into account design constraints as discussed earlier to be applicable in real-world applications. In this paper, we introduce a new design procedure to address these challenges.

It is also worth noting that two related tasks: hyperparameter optimization and network simplification/pruning are simpler tasks considering the dimension of the parameters compared with the whole architecture. In addition, a recent body of works on resources-aware network design such as MorphNet [12] and NetAdapt [30] have considered resource constraints in the design process. However, these techniques work on simplifying a pre-trained model to match the resource constraints, which is much simpler than an architecture search from scratch as being addressed in this paper.

3 Designing constraints

A network architecture, $\alpha$, is defined as a directed acyclic graph consisting of an ordered sequence of $L$ nodes. Each node, $x_{j}$, in the graph is a latent representation, a.k.a a feature map in the network. The first node is the input node, which is the input image. The final node is the output. Each intermediate node is computed as a summary of network operations applied on its predecessors,

[TABLE]

where $o^{(i,j)}$ is a candidate network operation with value taken from an operation set $\mathbb{O}$. The operation set can include a convolutional operator, $conv$; a pooling operator, $pool$; a skip connection operator, $Identity$; and a no connection indicator, $zero$. Each convolutional operator is followed by a batch norm operation by default. The size of the convolutional kernel can vary, e.g. $3\times 3$ and $3\times 5$. There are two types of convolution: traditional convolution and its dilated version [31]. The $Identity$ operation will function as a skip connection similar to the ResNet architecture [14], which adds the original signal from the input feature map to the output feature map. The $Zero$ operation will model the lack of connection between two feature maps.

With this notation, the network design is interpreted as two tasks: (1) searching for a set of network operations $\{o^{(i,j)}:\;j:1..L\;\;\text{and}\;\;i:1..j\;\;\}$; and (2) searching for the operation weights (a.k.a parameter values) to achieve the highest performance network in the iris recognition setting.

For each network, we consider two constraints:

• •

Compactness of model: number of parameters: $P$, which is directly related to memory required: $M_{P}$,

• •

Computation: number of FLOPS: $K$.

Our aim is to design a network that achieves the best accuracy conditioned on these constraints.

3.1 Re-interpreting layer connections

Considering $o^{(i,j)}$ is discrete, as shown in the literature, searching in the discrete space is extremely computational heavy and may result in missing the optimal point [13, 18]. We employ one adjustment to make the search space continuous by assigning a coefficient $\alpha_{o}^{(i,j)}$ for each candidate operations in the operation set $\mathbb{O}$. Other than activating a single operation while all others are disabled between two nodes $(i,j)$, we activate all candidate operations in the operation set but only one operation is strongly encouraged with a high coefficient value while others are strongly discouraged with small coefficient values. We call this a connection bundle as shown in Figure 1. In the traditional discrete case, $\alpha_{o}^{(i,j)}$ takes only one of two values, $\{0,1\}$. The most popular choice is one of $\alpha_{o}^{(i,j)}$ is $1$ while all others are [math]. In our continuous case, $\alpha_{o}^{(i,j)}$ can take any value in the range of $[0,1]$. This adjustment has been employed in the gradient-based architecture learning approaches [15].

The categorical summary in Equation 1 can now be interpreted as a softmax over all possible operations,

[TABLE]

where the denominator $Z$ is the normalization factor defined as $Z=\sum_{o^{\prime}\in\mathbb{O}}exp(\alpha_{o^{\prime}}^{(i,j)})$. After this relaxation, the task of architecture search reduces to learning a set of continuous variables $\alpha=\{\alpha^{(i,j)}\}$. At the end of the search, a discrete architecture can be obtained by replacing the pseudo operation $\overline{o}^{(i,j)}$ with the strongest operation, i.e. $o^{(i,j)}=\text{argmax}_{o\in\mathbb{O}}\;\;\alpha_{o}^{(i,j)}$.

One example of an architecture search space is illustrated in Figure 2.

3.2 Network design as a constrained optimization

Denote $L_{train}$ and $L_{val}$ as the training and validation losses respectively. Both losses are determined by not only the architecture, $\alpha$, but also the weights, $w$, of the network. The goal of our network design is to find $\alpha^{*}$ that minimizes the validation loss $L_{val}(w^{*},\alpha^{*})$, where the weight, $w^{*}$, associated with the architecture are obtained by minimizing the training loss, $w^{*}(\alpha)=\text{argmax}_{w}L_{train}(w,\alpha^{*})$.

Now the network design is a double learning problem. Taking the constraints mentioned before, the learning process can be represented as a bi-level constrained optimization problem as follows:

[TABLE]

[TABLE]

where the validation data is used to search for the architecture and the training data is used to search for the weights of an architecture. This is a two-level optimization problem where: (1) the lower level optimization problem is: searching for the best weights of the existing architecture to minimize the training loss; and (2) the upper level optimization problem is: searching for the best network architecture that, with its best weights, minimizes the validation loss.

This bi-level optimization has also arisen in hyperparameter optimization in the network design such as in [10] but in simple forms where the dimensions of the scalar-valued hyperparameters are substantially smaller than the architecture dimension.

3.3 Iterative solution

Solving the bilevel optimization exactly is prohibitive as it would require recomputing $w^{*}$ by solving the lower problem (6) whenever there is a change in $\alpha$. An iterative gradient-based solution has been shown to be effective [29]. However, popular gradient-based approaches such as Stochastic Gradient Descent (SGD) [2] is not applicable here due to the presence of inequality design constraints. To solve this, we propose an algorithm to solve this constrained bi-level optimization problem based on Projected Gradient Descent (PGD) [3] as in Algorithm 1. The gradient of the loss in the train subset, $\nabla_{w}L_{train}(w,\alpha)$, is used to update the weights, $w$, to learn the best weight for the current architecture. The gradient of the loss in the validation subset, $\nabla_{\alpha}L_{val}(w-\xi\nabla_{w}L_{train}(w,\alpha),\alpha)$, is used to guide the architecture update, where $\xi$ is a coefficient. The updated architecture, $\alpha$, is tested on the computation and memory constraints. If it does not satisfy the constraints, $\alpha$ will be projected into the constraint domain to generate a new architecture that satisfies the constraints. This iterative gradient-based approach has been applied in [15] but without constraints.

4 Experimental results

We have conducted our experiments on three publicly available datasets:

• •

ND-CrossSensor-Iris-2013 dataset 111https://sites.google.com/a/nd.edu/public-cvrl/data-sets: is the largest public iris dataset in the literature in terms of the number of images [21]. It contains 116,564 iris images captured by the LG2200 iris camera from 676 subjects.

• •

CASIA-Iris-Thousand dataset 222http://biometrics.idealtest.org: contains 20,000 iris images from 1,000 subjects, which were collected using the IKEMB-100 camera from IrisKing [4].

• •

UBIRIS.v2 iris dataset 333http://iris.di.ubi.pt/ubiris2.html: contains 11,102 iris images from 261 subjects with 10 images each subject. The images were captured under unconstrained conditions (at-a-distance, on-the-move and on the visible wavelength), with realistic noise factors [22].

Some examples and statistics of three datasets are presented in Figure 3 and Table I.

Three experiments will be performed for validation. First, the proposed algorithm will be employed to investigate the optimality of the existing approaches. By targeting the computation and/or model size of the existing approaches as the constraints for the search, our constrained search algorithm automatically discovers the network that achieves the highest achievable accuracy. Comparing this accuracy with the accuracy of the existing approaches, their optimality can be validated or rejected. The classic IrisCode and a recent deep iris network will be sequentially used for demonstration in the first two experiments presented in Section 4.3 and  4.4. The third experiment is to search for a network with competitive accuracy to the state-of-the-art approaches with less computation and model size requirements.

4.1 Experimental setup

We first pre-process the iris images by segmentation and normalization. The iris image is first segmented using two circles of the inner and outer boundaries of the iris region, detected by an integro-differential operator as,

[TABLE]

where $I(x,y)$ and $G_{\sigma}$ denote the input image and a Gaussian with a standard deviation $\sigma$, respectively. The symbol $*$ denotes a convolution operation and $r$ represents the radius of the circular arc $ds$, centered at the location $(x_{0},y_{0})$. This operation detects circular edges by iteratively searching for the maximum responses of a contour defined by the parameters $(x_{0},y_{0},r)$. We subsequently normalize the segmented iris regions to a fixed size by a rubber-sheet model [5]. This process is carried out by re-mapping the iris region, $I(x,y)$, from the raw Cartesian coordinates $(x,y)$ to the dimensionless polar coordinates $(r,\theta)$ as, $I(x(r,\theta),y(r,\theta))\rightarrow I(r,\theta)$, where $r$ is in the unit interval [0,1], and $\theta$ is an angle in the range of [0,2$\pi$]. $x(r,\theta)$ and $y(r,\theta)$ are defined as the linear combination of both pupillary $(x_{p}(\theta),y_{p}(\theta))$ and limbic boundary points $(x_{s}(\theta),y_{s}(\theta))$ as,

[TABLE]

This normalization step also helps to reduce the rotations of the eye (e.g., due to the head movement), to simple translation during matching. The corresponding noise mask is also normalized to facilitate matching in the later stages. For this segmentation and normalization, we used an open-source software, USIT v2.2, from the University of Salzburg [24] to segment, normalize and generate a normalized images with a fixed size of $64\times 512$ pixels.

4.2 Performance metrics

To report the performance, we rely on False Rejection Rate (FRR) and Equal Error Rate (EER). In this work, FRRs at False Acceptance Rate (FAR) = 0.1% are experimented and reported due to its popular adoption in the field.

State-of-the-art iris networks employ two types of losses: (1) cross-entropy loss [11, 19] and (2) pairwise loss [16, 32].

• •

Cross-entropy loss: the important property of the cross-entropy loss is using the same identities in the training and testing datasets, which means sample-disjoint but not subject-disjoint. This is shown in the softmax classifier of [11] and the SVM classifier of [19]. Hence to be comparable with the state of the art, we first divide our dataset into the sample-disjoint but not subject-disjoint training and testing subsets.

• •

Pairwise loss: the pairwise loss measures the similarity or dissimilarity between two input images, deciding whether they are from the same class or not. This loss allows us to have unseen subjects, i.e. not present in the training phase, in the test phase. This loss has been shown to be effective for iris recognition [16, 32]. Hence we also split the data into subject-disjoint training and testing subsets.

In summary, there are two data splitting schemes in this work: (1) sample-disjoint splitting and (2) subject-disjoint splitting. Depending on the task to be performed, our experiments will apply one of these two schemes. For the sample-disjoint scheme, we split images of each subject into 70% of the images for training, 10% of the images for validation and 20% of the images for testing. For the subject-disjoint scheme, we split 70% of the subjects for training, 10% of the subjects for validation and 20% of the subjects for testing.

Intra-dataset performance We perform the intra-dataset experiment on the ND-CrossSensor-Iris-2013 dataset, not the other two datasets, due to its large size suitable for training. The training subset is to train a network architecture to find the best weights. The validation subset is used to find the best network architecture. The testing subset is used to report the intra-dataset performance.

The classic IrisCode is used as a handcrafted baseline. Our implementation achieves a $FRR=3.76\%$ at $FAR=0.1\%$ and an $EER=1.75\%$ on the ND-CrossSensor-2013 dataset, which is comparable to the state of the art implementation [32].

Cross-dataset performance The best network learned is further investigated for generalization capability through training in one dataset and testing on others. The best network discovered in the ND-CrossSensor-Iris-2013 dataset is tested on the other two datasets, CASIA-Iris-Thousand and UBIRIS.v2 to understand its generalizability. We do not perform network search on the CASIA and UBIRIS datasets as their small number of images will restrict the search space.

4.3 Case 1: Handcrafted - IrisCode

Firstly, we are interested to see how well the deep networks perform if they have to limit their computation akin to the one in the classic handcrafted IrisCode [6]. We impose one computation constraint, i.e. the maximum number of FLOPs, to be akin to the one in the classic IrisCode, and investigate the best accuracy a deep network could achieve compared to the accuracy of the IrisCode. We run the constrained design algorithm to find the best network architecture yielding the highest accuracy or smallest EER conditioned on the IrisCode computation.

Operation set We apply popular operations, which are widely used in the existing deep iris networks, in the operation set $\mathbb{O}$: $3\times 3$ and $3\times 5$ convolution, $2\times 2$ max pooling, $2\times 2$ average pooling, $Identity$ and $Zero$. All operations are of stride 1.

Loss choice We apply the most popular softmax classifier with a classification cross-entropy loss. A majority of previous deep iris recognition networks use this classification loss trained over a set of known iris identities and then take the intermediate bottleneck layer as a representation beyond the set of identities used in training. We use the classification probabilities as scores on known subjects in the test set. Varying the score threshold generates different operating points in the ROC curve. To be succinct, we only present the EER in the first two experiments. The ROC curve will be presented in the third experiment when comparing with the state-of-the-art approaches.

Dataset The experiment is performed on the ND-CrossSensor-Iris-2013 dataset. Due to the cross-entropy loss, the sample-disjoint scheme is applied to split the dataset into 70% of the images in each subject for training, 10% of the images in each subject for validation and 20% of the images in each subject for testing.

The constrained design task with input constraints, hyperparameters on the operation set and the loss choice, and the outputs are summarized in Table II. We found that the best network constrained by the number of FLOPs of the classic IrisCode achieves an $EER=16.7$, which is 10 times the $EER$ achieved by the IrisCode. Two interesting points can be inferred here: (i) deep networks struggle when the computation is strictly limited. Fundamentally, strictly limited computation is detrimental to the modeling capacity and learning algorithms; and (ii) this reinforces the effectiveness of the classic IrisCode in terms of both computation and recognition accuracy.

We subsequently pose a question related to how much additional computation and memory we have to sacrifice to achieve similar or even better accuracies than the handcrafted counterpart. We gradually increased the constraint $K$ values by a $K_{IrisCode}$ step and the initial number of layers, then reported the EERs as shown in Table III. The same level of accuracy is only achieved with $8$ times the additional computation amount with the network architecture discovered in Figure 4. This aligns with the well-known universal approximation theory of deep networks [9], which states that a deep network can approximate any function given enough resources. This illustrates the main characteristic of these deep networks: automatic feature engineering comes at the cost of heavy computation and memory requirements.

4.4 Case 2: Deep learning - ResNet18

Secondly, we are interested to see how effective the state-of-the-art deep networks perform in terms of architecture design. In other words, under the same computation and memory with the existing state-of-the-art networks, what is the best accuracy we can achieve for the iris recognition task. Nguyen et al. analyzed layer by layer performance of the landmark networks which have won the ImageNet challenge since 2012 to the iris recognition tasks [19]. Despite being pre-trained on the ImageNet, they have shown competitive performance on the iris recognition task with transfer learning. We choose one of the landmark networks, called ResNet18, for experiments due to its simplicity and uniformity in the layer connections and wide adoption in the field [14]. We apply the proposed constrained design algorithm to search for the network with the best accuracy that can be yielded bounded by the computation, $K_{ResNet18}$, and the number of parameters, $P_{ResNet18}$, of the ResNet18 network. We employ the same operation set, the cross-entropy loss and the data splitting scheme as in Case 1. The constrained design task is summarized in Table IV. Our algorithm discovers a new architecture that achieves higher accuracy than the original ResNet, $EER=1.12\%$ vs. $1.29\%$ and $FRR=2.23\%$ vs. $2.58\%$ with the same level of computation and memory.

4.5 Case 3: State-of-the-art

We also want to see whether we can achieve competitive accuracy compared with the state-of-the-art deep iris networks.

Operation set We apply popular operations in the operation set $\mathbb{O}$: $3\times 3$ and $3\times 5$ convolution, $3\times 3$ and $3\times 5$ dilated convolution, $2\times 2$ max pooling, $2\times 2$ average pooling, $Identity$ and $Zero$. All operations are of stride 1.

Loss choice We leverage the most recent advance in the loss design for biometrics by using a pairwise loss called Extended Triplet loss as investigated in [28, 32]. Compared with classification losses, pairwise losses directly reflect what we want to achieve, i.e. to train the representation to correspond to iris similarity. This results in irises of the same person having small distances and irises of different people having larger distances.

Dataset The experiment is performed on the ND-CrossSensor-Iris-2013 dataset. Due to the pairwise loss, the subject-disjoint scheme is applied to split the dataset into 70% of the subjects for training, 10% of the subjects for validation and 20% of the subjects for testing.

The constrained design task is summarized in Table V. Running on a single Nvidia GTX 1080Ti GPU, our algorithm discovered a network as presented in Figure 5 in 26 hours. We compare the discovered network with the state-of-the-art approaches in four metrics: two in accuracy ($FRR$ and $EER$), one computation ($K$) and one memory ($P$).

We compare with two state-of-the-art deep iris networks using pairwise loss including: DeepIris [16] and FeatNet [32]. Since the original models in their papers are not publicly available, we carefully implemented and trained the networks according to all the details in [16, 32]. A notable difference is we use the same segmentation method from USIT v2.0 for all approaches to be comparable. The normalized iris images, with a size of $64\times 512$, are fed directly to [32] since they use the same input size. The normalized iris images are resized to the expected size of [16, 32] to be compatible with their network designs. The performance achieved is comparable to those reported in the papers. Table VI and the ROC curve in Figure 6 show the discovered network outperforms all existing deep iris networks in terms of accuracy with less number of parameters and computation required.

4.6 Generalizability of the model

Finally, we want to understand the generalization capability of the network architecture discovered by performing a cross-dataset experiment on two smaller-size datasets, CASIA [4] and UBIRIS [22]. While the CASIA dataset captured the NIR iris images using a different camera, the UBIRIS dataset captured the iris images with a visible camera. This demonstrates the wide range of imaging conditions to test the generalizability. Two datasets CASIA and UBIRIS are splitted into 20% for training and 80% for testing. Three networks, i.e.: ours, DeepIris and Featnet, trained as per Section 4.5 are further fine-tuned using the training subset and tested in the testing subset of the two datasets for cross-dataset performance investigation.

The performance is presented in Table VII. The network discovered by our architecture search algorithm outperforms the state-of-the-art approaches in both CASIA and UBIRIS datasets, illustrating a high level of generalization across different sensors, different imaging distances and different levels of subject cooperation.

5 Conclusions

This paper proposes an algorithm to design a deep iris recognition network with attention to computation and memory constraints. By modeling the design process as a bi-level constrained optimization approach, our algorithm is able to search for the optimal network which achieves the best possible performance conditioned on the pre-defined computation and model compactness constraints. This algorithm enables us to investigate the effectiveness of the classic handcrafted IrisCode compared with deep network counterparts. It also enables us to further improve the existing deep iris recognition networks to achieve similar or better accuracy with the same level of computation and memory cost. The design algorithm also discovers an optimal network with competitive performance with less computation and memory required than the state-of-the-art approaches, in both intra-dataset and cross-dataset experiments. More importantly, this algorithm simultaneously achieves both automatic feature engineering and network architecture engineering, opening us to full automation in deep iris recognition network design.