Relevant features for Gender Classification in NIR Periocular Images

TL;DR

This paper identifies the periocular region in NIR images as more informative for gender classification than the iris itself, demonstrating that focusing on this area improves accuracy and speed.

Contribution

It analyzes and demonstrates the most relevant features in the periocular region for gender classification, introducing a new database and emphasizing the importance of the periocular area over the iris.

Findings

Periocular region contains more gender information than iris.

Best classification accuracy (89.22%) achieved with 4,000 features.

Focusing on periocular area enables faster gender classification.

Abstract

Most gender classifications methods from NIR images have used iris information. Recent work has explored the use of the whole periocular iris region which has surprisingly achieve better results. This suggests the most relevant information for gender classification is not located in the iris as expected. In this work, we analyze and demonstrate the location of the most relevant features that describe gender in periocular NIR images and evaluate its influence its classification. Experiments show that the periocular region contains more gender information than the iris region. We extracted several features (intensity, texture, and shape) and classified them according to its relevance using the XgBoost algorithm. Support Vector Machine and nine ensemble classifiers were used for testing gender accuracy when using the most relevant features. The best classification results were obtained when 4,000 features located on the periocular region were used (89.22\%). Additional experiments with the full periocular iris images versus the iris-Occluded images were performed. The gender classification rates obtained were 84.35\% and 85.75\% respectively. We also contribute to the state of the art with a new database (UNAB-Gender). From results, we suggest focussing only on the surrounding area of the iris. This allows us to realize a faster classification of gender from NIR periocular images.

Tables

Table 1. TABLE I: Summary of gender classification using eyes: I = Iris Images, P = Periocular Images, L = Left and R = Right, FEM = Feature Extraction Method, CCR= Correct Classification Rate.

Paper	I/P/F	Source	$N^o$ of Images	$N^o$ of Subjects	Type	FEM	CCR (%)
Thomas et al. [24]	I	Iris	16,469	N/A	NIR	Gabor Filter	75.00
Lagree et al.[25]	I	Iris	600	300	NIR	vGabor Filter	62.17
Bansal et al. [26]	I	Iris	400	200	NIR	Discrete wavelet transform	83.60
Tapia et al. [13]	I	Iris	1,500	1,500	NIR	LBP	91.00
Costa et al. [27]	I	Iris	1,600	200	NIR	Shape+Texture Features, Gabor Filter	89.74
Bobeldyk et al.[28]	I / P	Iris	3,314	1,083	NIR	BSIF	85.70 (P)
							65.70 (I)
Merkow et al. [1]	P	Faces	936	936	VIS	LBP	80.00
Chen et al. [29]	P	Faces	2,006	1,003	NIR/Thermal	LBPH	93.59
[30]	P	Faces	3,000	1,500	VIS	HOG,LBP,LTP	92.46
						WLD,LOSIB
Kuehlkamp et al. [31]	I	Iris	3,000	1,500	NIR	LBP	66.00
Rattani et al. [9]	P	Faces	572	200	VIS	LBP,LPQ,BSIF	91.60
Tapia et al. [23]	I	Iris	10,000 unlabel	–	NIR	RBM	77.79
			3,000 labeled	1,500		CNN	83.00
Kuehlkamp et al. [32]	P	Iris	6,240	2005	NIR	Hand-Crafted,CNN	80.80
This paper	P	Iris/Faces	952 (P-UND L)	952 (P-UND L)	NIR	RAW,ULBP,HOG	85.70
			946 (P-UND R)	946 (Pe-UND R)
			3,840 (C.E.)	120 (C.E.)
			389(UTIRIS)	79 (UTIRIS)
			1,360(UNAB-Gender)	136 (UNAB-Gender)
This paper							89.22 (4,900)
With XgBoost

Table 2. TABLE II: Correct classification rates for the left and right periocular images trained and tested using UND Periocular considering the comparison between the Non occluded and Occluded iris images.

Method	Test 1		Test 2
	Left eye	Right eye	Left eye	Right eye
	Non occluded	Non occluded	Occluded	Occluded
Experiment 1
Raw	53.57+/-0.7	51.57 +/- 0.7	54.57 +/- 0.7	53.57+/- 0.7
Experiment 2
ULBP (8,1)	73.68 +/- 0.6	78.83+/- 0.6	75.78 +/- 0.8	75.66 +/- 0.8
ULBP (8,2)	74.73 +/- 0.6	73.80 +/- 0.7	76.31 +/- 0.8	75.92 +/- 0.8
ULBP (8,3)	74.73 +/- 0.7	74.07 +/- 0.7	75.78 +/- 0.7	74.60 +/- 0.6
ULBP (8,4)	74.73+/- 0.6	74.60 +/- 0.7	79.47 +/- 0.7	74.60 +/- 0.6
ULBP (8,5)	76.31 +/- 0.6	78.83 +/- 0.6	78.42 +/- 0.7	78.30 +/- 0.6
ULBP (8,6)	77.89 +/- 0.6	74.60 +/- 0.6	76.31+/- 0.7	77.24,+/- 0.5
ULBP (8,7)	74.21 +/- 0.6	73.80 +/- 0.6	75.52 +/- 0.7	77.24,+/- 0.5
ULBP (8,8)	74.73 +/- 0.6	77.24+/- 0.6	76.31+/- 0.7	75.66 +/- 0.6
Experiment 3
ULBP (8,1- 8,8)	78.68 +/- 0.6	80.42 +/- 0.6	78.82 +/- 0.7	84.33 +/- 0.6
Experiment 4
HOG (3x3)	78.42 +/- 0.6	82.01 +/- 0.6	76.57 +/- 0.7	79.01 +/- 0.8
HOG (5x5)	82.83 +/- 0.5	83.06+/- 0.5	74.73 +/- 0.4	78.04 +/- 0.7
HOG (10x10)	80.52 +/- 0.6	85.74 +/- 0.4	80.00 +/- 0.5	83.59 +/- 0.6
Experiment 5
Fusion	81.84	83.60	82.63	82.01

Table 3. TABLE III: Correct classification rates with method HOG/LBP for nine ensemble classifier applied to the best results of Table II . B represents: Boost.

Classifier	Left eye (%)	Right eye (%)	Left eye (%)	Right eye (%)
	Non occluded	Non occluded	Occluded	Occluded
SVM	82.83 +/- 0.5	85.74 +/- 0.4	80.00 +/-0.5	84.33 +/- 0.6
Bag	79.52 +/- 0.3	84.66 +/- 0.3	81.33 +/- 0.3	82.33 +/- 0.6
Ada-B	78.66 +/- 0.6	84.33 +/- 0.6	81.33 +/- 0.6	81.66 +/- 0.6
Logit-B	80.33 +/- 0.3	79.66 +/- 0.6	80.66 +/- 0.3	83.33 +/- 0.3
Gentle-B	81.00 +/- 0.3	79.66 +/- 0.3	79.66 +/- 0.6	80.66 +/- 0.6
Robust-B	79.66 +/- 0.3	82.33 +/- 0.6	80.66 +/- 0.3	81.33 +/- 0.3
LP-B	81.66 +/- 0.6	83.33 +/ 0.3	81.00 +/- 0.3	82.33 +/- 0.6
Total-B	79.66 +/- 0.6	81.33 +/- 0.6	79.66 +/- 0.6	80.33 +/- 0.3
Rus-B	78.66 +/- 0.3	81.66 +/- 0.3	78.66 +/- 0.6	80.66 +/- 0.3
RF GDI	81.33 +/- 0.6	84.66 +/- 0.5	81.66 +/- 0.6	84.00 +/- 0.3

Table 4. TABLE IV: Correct classification rates for the left and right periocular images trained with UND database (Non occluded and Occluded) and tested with UTIRIS NIR .

Method	Test 1		Test 2
	Left eye	Right eye	Left eye	Right eye
	Non occluded	Non occluded	Occluded	Occluded
Experiment 1
Raw	55.60+/-0.6	57.53 +/- 0.5	55.90 +/- 0.8	58.00+/- 0.3
Experiment 2
ULBP (8,1)	80.55 +/- 1.0	83,80+/- 1.0	85.09 +/- 0.8	85.35 +/- 0.8
ULBP (8,2)	83.55 +/- 0.9	79.18 +/- 0.9	85.09 +/- 0.8	66.32+/- 0.9
ULBP (8,3)	83.80 +/- 0.8	80.46 +/- 0.8	84.32 +/- 0.7	70.44 +/- 0.7
ULBP (8,4)	80.98 +/- 0.9	68.64 +/- 0.8	76.91+/- 0.7	82.78 +/- 0.6
ULBP (8,5)	83.03 +/- 0.8	66.84 +/- 0.8	79.18 +/- 0.7	84.83 +/- 0.6
ULBP (8,6)	82.01 +/- 0.8	85.35 +/- 0.6	82.26+/- 0.7	82.26 +/- 0.6
ULBP (8,7)	81.75 +/- 0.8	81.32 +/- 0.7	81.49 +/- 0.7	77.63 +/- 0.6
ULBP (8,8)	81.75 +/- 0.6	77.12+/- 0.7	53.21+/- 0.7	74.04,+/- 0.6
Experiment 3
ULBP (8,1- 8,8)	84.32+/- 0.5	84.58 +/- 0.6	84.32 +/- 0.7	64.27+/- 0.6
Experiment 4
HOG (3x3)	71.12 +/- 0.6	72.24 +/- 0.6	69.92 +/- 0.7	58.61 +/- 0.6
HOG (5x5)	57.58 +/- 0.6	65.04+/- 0.6	41.13 +/- 0.7	47.59+/- 0.6
HOG (10x10)	52.19 +/- 0.6	60.33 +/- 0.5	56.22 +/- 0.7	57.22+/- 0.6
Experiment 5
Fusion	85.60	82.26	85.26	86.89

Table 5. TABLE V: Correct classification rates for the left and right periocular images trained with UND database (Non occluded and Occluded) and tested with Cross-Eye-2016 NIR .

Method	Test 1		Test 2
	Left eye	Right eye	Left eye	Right eye
	Non occluded	Non occluded	Occluded	Occluded
Experiment 1
Raw	60.61+/-0.3	59.30 +/- 0.5	59.33 +/- 0.6	59.00+/- 0.6
Experiment 2
ULBP (8,1)	68.96 +/- 0.9	69.38+/- 0.8	72.40 +/- 0.8	74.69 +/- 0.8
ULBP (8,2)	68.94 +/- 0.7	74.48 +/- 0.7	70.63+/- 0.8	66.98+/- 0.9
ULBP (8,3)	72.40 +/- 0.6	79.90 +/- 0.6	78.85 +/- 0.7	74.90 +/- 0.7
ULBP (8,4)	77.19 +/- 0.9	52.08 +/- 0.9	77.40 +/- 0.7	71.88 +/- 0.6
ULBP (8,5)	72.67 +/- 0.6	68.65 +/- 0.6	80.21+/- 0.9	72.29 +/- 0.6
ULBP (8,6)	77.50 +/- 0.6	77.29 +/- 0.6	75.00+/- 0.7	74.58 +/- 0.6
ULBP (8,7)	77.50 +/- 0.7	78.23 +/- 0.7	79.48 +/- 0.5	76.35 +/- 0.6
ULBP (8,8)	78.85 +/- 0.6	79.90 +/- 0.9	77.92+/- 0.7	75.10 +/- 0.6
Experiment 3
ULBP (8,1- 8,8)	75.10+/- 0.6	76.88 +/- 0.6	76.04 +/- 0.7	73.96+/- 0.6
Experiment 4
HOG (3x3)	73.54 +/- 0.9	71.56 +/- 0.6	75.10 +/- 0.7	65.10 +/- 0.6
HOG (5x5)	67.08 +/- 0.6	74.79 +/- 0.6	57.92 +/- 0.7	51.88+/- 0.6
HOG (10x10)	74.58 +/- 0.6	63.23 +/- 0.5	55.21 +/- 0.9	57.29+/- 0.6
Experiment 5
Fusion	77.71	77.92	70.73	71.77

Table 6. TABLE VI: Correct classification rates for the left and right periocular images trained with UND database (Non occluded and Occluded) and tested with UNAB-Gender .

Method	Test 1		Test 2
	Left eye	Right eye	Left eye	Right eye
	Non occluded	Non occluded	Occluded	Occluded
Experiment 1
Raw	54.61+/-0.66	53.30 +/- 0.3	53.66 +/- 0.6	55.00+/- 0.6
Experiment 2
ULBP (8,1)	71.66 +/- 0.9	72.33+/- 0.8	72.66 +/- 0.8	73.66 +/- 0.8
ULBP (8,2)	71.66 +/- 0.7	74.48 +/- 0.7	73.66+/- 0.8	72.33+/- 0.9
ULBP (8,3)	72.40 +/- 0.6	76.90 +/- 0.6	78.33 +/- 0.8	74.33 +/- 0.5
ULBP (8,4)	77.33 +/- 0.9	72.33 +/- 0.9	77.33 +/- 0.3	72.66 +/- 0.6
ULBP (8,5)	74.66 +/- 0.3	78.65 +/- 0.3	78.21+/- 0.6	76.66 +/- 0.6
ULBP (8,6)	77.33 +/- 0.6	73.33 +/- 0.6	74.66+/- 0.7	76.66 +/- 0.6
ULBP (8,7)	77.33 +/- 0.7	75.66 +/- 0.7	76.33 +/- 0.5	76.33 +/- 0.3
ULBP (8,8)	76.66 +/- 0.4	80.33 +/- 0.3	75.33+/- 0.7	76.33 +/- 0.3
Experiment 3
ULBP (8,1- 8,8)	79.66+/- 0.3	78.66 +/- 0.6	76.33 +/- 0.7	83.33+/- 0.6
Experiment 4
HOG (3x3)	77.33 +/- 0.6	80.66 +/- 0.6	75.33 +/- 0.6	75.66 +/- 0.6
HOG (5x5)	81.33 +/- 0.3	80.33 +/- 0.6	79.33 +/- 0.6	81.66+/- 0.6
HOG (10x10)	81.66 +/- 0.6	82.33 +/- 0.5	77.66 +/- 0.6	80.66+/- 0.6
Experiment 5
Fusion	81.0	79.6	80.3	81.6

Table 7. TABLE VII: Correct classification rates for training the left and right periocular images with a training set of UND+UTIRIS+Cross-Eyed databases and testing with a test set of UND+UTIRIS+Cross-Eyed. TNR: True Negative Rate. TPR: True Positive Rate, MCC: Matthews Correlation Coefficient .

Method	Test 6
	Left eye (%)				Right eye (%)
	CCR	TPR	TNR	MCC	CCR	TPR	TNR	MCC
Experiment 1
Raw	65.33	67.33	64.33	0.44	64.33	64.33	62.33	0.45
Experiment 2
ULBP(8,1)	81.46	81.11	63.87	0.46	82.79	87.43	76.40	0.62
ULBP(8,2)	78.47	82.11	64.43	0.51	80.56	85.54	72.75	0.57
ULBP(8,3)	76.25	80.97	62.75	0.47	76.02	82.38	67.13	0.47
ULBP(8,4)	77.41	82.39	66.11	0.50	81.72	86.66	75.00	0.60
ULBP(8,5)	79.15	84.19	70.03	0.54	81.43	86.05	73.60	0.59
ULBP(8,6)	76.35	82.20	66.39	0.48	79.98	85.10	71.91	0.56
ULBP(8,7)	77.70	82.46	66.11	0.50	78.92	82.39	64.89	0.52
ULBP(8,8)	77.22	82.43	66.39	0.49	78.05	82.45	65.73	0.51
Experiment 3
ULBP (8,1- 8,8)	83.49	84.59	70.35	0.56	83.56	87.46	76.12	0.64
Experiment 4
HOG (3x3)	79.63	84.11	69.47	0.55	82.79	86.76	74.72	0.62
HOG (5x5)	82.14	84.59	69.11	0.60	82.59	85.27	70.79	0.61
HOG (10x10)	80.41	84.60	68.35	0.63	81.72	84.68	69.66	0.59
Experiment 5
Fusion	79.79	80.63	79.38	0.52	80.33	92.70	77.02	0.57

Table 8. TABLE VIII: Summary of the best results from table II to table VII . The third Column shows a MCC for the each experiment.

Test	Method	MCC
Test 1	HOG(10x10)	0.61
Test 2	ULBP(8,1-8,8)	0.60
Test 3	Fusion	0.62
Test 4	ULBP(8,5)	0.54
Test 5	HOG(10x10)	0.59
Test 6	ULBP(8,1-8,8)	0.64

Table 9. TABLE IX: Features selected from Non occluded versus occluded UND dataset using the best results with XgBoost algorithm from Table II and different thresholds values. N= Number of features selected, CCR= Correct Classification Rate.

LEFT EYE			RIGHT EYE			LEFT EYE			RIGHT EYE
Non Occluded						Occluded
Thresh.	N	CCR.%	Thresh.	N	CCR.%	Thresh.	N	CCR.%	Thresh.	N	CCR.%
0.00000	9,000	81.27	0.00000	9,000	85.94	0.00000	9,000	78.41	0.00000	9,000	84.35
0.00151	2,900	81.27	0.00152	2,900	85.94	0.00156	2,900	78.41	0.00148	3,020	84.35
0.00452	7,900	82.22	0.00759	2,900	85.94	0.01252	1,100	80.95	0.00890	2,100	79.55
0.00754	2,800	81.90	0.01366	1,000	80.83	0.00782	3,200	81.59	0.00742	3,200	81.15
0.00302	1,470	83.90	0.00303	1,290	86.58	0.00313	1,400	79.37	0.01039	1,900	80.51
0.01056	1,500	78.41	0.0067	4,900	89.22	0.00469	8,200	80.00	0.00593	5,000	82.75
0.01207	8,000	76.83	0.00455	8,100	87.22	0.01095	1,600	81.00	0.00455	7,200	83.07
0.00603	5,300	70.79	0.00910	1,900	84.35	0.00939	2,100	79.41	0.00297	1,300	83.39

Equations

$$LB{P}_{P,R}(x,y)=\bigcup _{(x^{\prime },y^{\prime })\in N(x,y)}h(I(x,y),I(x^{\prime },y^{\prime }))$$

$$\|x\|:=\bigg{(}\int x^{2}(t)dt\bigg{)}^{1/2}=\bigg{(}\sum_{i=1}^{L}x_{i}^{2}\bigg{)}^{1/2},$$

Full text

I Introduction

Periocular biometrics have recently attracted much attention since it allows improvement of the robustness of face or iris biometric recognition. The periocular region is the area surrounding the eye, and it is generally considered one of the most discriminative regions of the face [1]. It has been shown that the periocular region itself can be used for recognition [2]. It can also help with iris recognition when the inherent biometric content within the source images of poor quality. This suggests that periocular features could have a potential for soft biometric classification.

Recently, Soft-biometrics studies have proposed the use of iris information for estimating demographic information such as gender, ethnicity, age, and emotions. [3, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12]. Gender information, in particular, can improve biometric recognition administration systems, since it allows reduced search time [13]. This is especially important in countries such as India, China and others which have large populations [14, 15, 16]. For instance, if gender is determined then the average search time may be halved.

Gender information may also be useful when people are not recognized but is attempting to gain entry to a restricted zone. Another possible use of the gender information arises in social settings, where it may be useful to screen entry to an area based on the gender of a person, while not recording identity [17]. Gender classification is also crucial for banking transactions from cellular phones applications, demographic information collection, marketing research, and real-time digital marketing [18, 19, 8]. Therefore, this information is essential to manage iris recognition systems better.

Most gender classification methods reported in the literature use the entire face as Region of Interest (ROI) [1]. However, the performance of these methods suffers greatly when portions of the face are occluded. Recent studies have narrowed the principal region for biometric identification to the surrounding eye area. This area is known as the periocular region [20].

Most periocular algorithms work in a holistic fashion. They usually define the rectangle that contains the whole eye area as an ROI. The full rectangular area is then used for feature extraction. Such a holistic approach implies that some components that are not relevant for identity recognition, like hair or glasses, may erroneously bias the recognition process. Furthermore, features may not be equally discriminative in all parts of the periocular regions.

Periocular biometrics techniques have mainly focused on texture features. These features are calculated from a single periocular image. Therefore, can not efficiently handle variations introduced by movements of the eyeball and eyelid. The performance further suffers if we add variations in pose and illumination.

Hollingsworth et al. [21] identified some of the ocular elements that are found most useful for periocular recognition. The relevance of those elements depends on the technology used for capturing the image. For Near-Infra-Red (NIR) images the most influential elements are the tear duct and lower lash. The skin and cheek are the least relevant. When using Visual Spectrum (VIS) images, on the contrary, skin texture is shown to be more important as well as the blood vessels and the brown region. Those results agree with assumptions made in the literature indicating NIR illumination is more suitable for iris texture recognition [22] while VIS illumination performs better for analyzing the periocular region. Several studies have been carried out using automatic algorithms with similar results.

We have previously demonstrated that gender classification based on iris information (NIR images) achieves similar results as classical method using facial biometric features [17, 23].

In this paper, we analyze the relevance of the different areas around the eye. Using the same periocular NIR images used in iris-identification systems, we investigated the relevance of the iris on classifying gender in an effort to identify the best feature extraction techniques to increase gender classification rates. We also question whether the fusion of iris and periocular information is complementary. Further we contribute two new gender labeled databases, which we captured for this study.

This paper is organized as follows. In Section 2, we review the state of the art in gender classification. In Section 3, we describe the proposed method and present the feature extraction techniques used. In section 4, we provide details about the datasets used in our experiments. In Section 5, we describe the experiments and results, and in Sections 6 and 7, we present the discussion and conclusions, respectively.

II State of the Art

Gender classification initially used a variety of features extracted from facial areas. [19, 8] reported the use of a feature-selection method based on mutual information and feature fusion to improve the gender classification of facial images.

Castrillon-Santana et al. [30] proposed a gender-classification system that works with periocular images. This area is extracted after normalizing the face in terms of scale and rotation. Given the rough eye-location, the normalized facial image is obtained automatically after rotating, scaling, and cropping the original images. The best result was obtained using five different feature-extraction methods. Also, some studies have made use of annotated data, for the detection and segmentation of the periocular region of the eye [2]. This technique has become a research target itself.

Alonso-Fernandez et al. [2] report a survey with the most commonly used techniques based on periocular images and databases. They provide a comprehensive framework covering different perspectives, from existing databases to algorithms for the detection of the periocular region and features for recognition. Databases that were utilized include face and iris images (since the periocular area appears in such data), as well as newer databases that specifically capture the periocular area [2].

Kumari et al. [33] propose a novel approach of extracting global features from the periocular region of poor-quality grayscale images for gender classification. In their approach, global gender features are extracted using independent component analysis and evaluated using conventional neural-network techniques. The authors compare the performance, and all the best results are coming from periocular region cropped from the FERET face database [34].

Studies based on NIR normalized or encoded images of the iris have explored the possibility of automatically deducing the gender of an individual [26, 25, 24, 13, 28, 17]. Previous research on this topic has extracted and used the iris region, while most operational iris biometric systems typically acquire the periocular region for processing. Very few databases have been designed specifically for gender-classification from NIR periocular research.

Bobeldyk et al. [28] explored the correct gender-classification rate associated with four different regions from NIR iris images: the periocular region, the iris-excluded ocular region, the iris-only region, and the normalized iris-only region. They used a binarized statistical image feature (BSIF) texture operator to extract features from the regions that were previously defined. The ocular region reached a correct classification rate of 85.7%, while the normalized or unwrapped images exhibited the worst correct classification rate at only 65%. This suggests that the normalization process may filter out useful information.

Kuehlkamp et al. [32], Very recently, analyses the influence of the feature extracted from normalized and periocular images to classify gender in a larger database. The authors used different probabilistic occlusion masking to gain insight into the problem. Results also suggest the discriminative power of the iris texture for gender is weaker than previously thought, and that the gender-related information is primarily in the periocular region. Regardless of the classifier (SVM or CNN), there is a clear and significant difference between predicting gender from periocular images and normalized iris images. A summary of the most relevant work in this area is presented in Table I.

III Methods

Traditional iris-identification systems have five stages: acquisition, segmentation, normalization, encoding, comparison, and decision making. Periocular NIR images can be obtained from the first stage of acquisition. See Figure 1.

n this work, we directly used the acquired image from a conventional NIR sensor for extracting the features used in gender-classification. Despite other methods that require pre-processing stages such as Segmentation, Normalization, Encoding and Comparison (See Figure 1).

Intensity, shape, and texture information from periocular images were used to evaluate the gender classification performance. There is no need for additional steps to achieve the soft-biometrics information.

We used four different types of iris features obtained from periocular images to classify gender: intensity (raw image), texture features (Uniform Local Binary Patterns (ULBP)), shape (histogram of oriented gradient (HOG) and inverse HOG (IHOG).

For all datasets the image size were $120\times 160$ pixels (See Figure 3). Along of this research, we have done a preliminary test using the original sizes of the images, but the results were very similar when the size is reduced. Since the resolution did not affect our correct classification rate we keep doing all the experiments with $120\times 160$ pixels.

III-A Intensity images

As intensity, we used grayscale pixel values of NIR images. Eight bits per pixel were used. The UND periocular database was captured by a regular LG 4000 NIR sensor. The UTIRIS Database [35] used a ISG Lightwise LW camera. The NIR Cross-Eyed Databases [36], on the other hand, used a custom developed sensor that can capture simultaneously the NIR and RGB image. Therefore, the conditions and the distances while capturing images are different.

III-B Texture - Uniform Local Binary Patterns (ULBP)

The texture feature was extracted from LBPs as a gray-scale texture operator. It characterizes the spatial structure of the local image texture [37]. Given a central pixel in the image, a binary pattern number was computed by comparing its value with those of its neighbors. The original operator used a 3x3 window size that contains nine values. We computed the LBP features from the pixel intensities in a neighborhood.

[TABLE]

Where $N(x,y)$ is the vicinity around $(x,y)$, $\cup$ is the concatenation operator, $P$ is the number of neighbors, and $R$ is the radius of the neighborhood.

Ojala et al. [37] defined a Uniform Local Binary Pattern (ULBP) to improve the traditional approach using all circular transitions $2^{8}=256$. The uniform mapping produces only 59 output labels for neighborhoods when having eight sampling points. An LBP is called uniform when its uniformity measure is of 2 transitions. For instance, pattern 01110000 shows 2 transitions so it is uniform. The texture recognition improves when using uniform patterns of (8, 1) neighborhood [38]. This was also shown for gender classification in faces [8, 39](See Figure 2).

III-C Shape - Histogram of gradients

The shape features were extracted from the HOG and inverse HOG using three different scales: $3\times 3$, $5\times 5$, and $10\times 10$ [40, 41]. Vertical and horizontal edge maps were computed using the masks $\left[-1,0,1\right]$ and $\left[-1,0,1\right]^{T}$. We considered $v$ and $h$ to be the vertical and horizontal edge values at any pixel. They were obtained by the convolution of the edge mask with the original image respectively. The edge map was computed using $\theta=tan^{-1}\left(\frac{v}{h}\right)$, and the edge magnitude was computed as $m=\sqrt{v^{2}+h^{2}}$. The edge map was discretized at 18*∘* intervals. Each pixel contribute its magnitude $m$ to the bin that corresponded to its edge directions $\theta$. An image is represented by $20\times N$ reals values. Where $N$ is the number of windows (See Figure 2).

IV Databases

One of the most challenging problems in gender-classification when using iris or periocular images is the lack of a dataset for the training process. Most of the databases were created with a focus on the iris/face identification problem. Therefore, databases with gender information are private and usually not available for research. The databases usually do not have enough subjects.

In this paper, we use five databases that include gender information. The first one is a subset from the GFI-UND [17]. This database was captured using an LG-4000 iris sensor with full periocular images for the left and right eyes. Examples are shown in Figure 3a. The UND periocular Dataset is a new dataset different than used in [17]. This dataset will be available to other researchers upon request.

The second one is the same database, but with the iris masked in black for the left and right eyes. Examples are shown in Figure 3b. Those two datasets are person-disjoint and have one left eye image and one right eye image for each male and female. Note that a disjoint database assures that iris images are only present in training or the testing partition, but not in both.

The left full eye dataset has a partition of 952 subjects (462 female images and 490 male images). The right full eye dataset, on the other hand, has 946 subjects (458 female images and 488 male images). Six images were removed from the dataset because they caused errors on the automatic segmentation mask. https://www.overleaf.com/project/5c1a5f3c3235f057efbfb5a9

The third database is the University of Tehran IRIS database (UTIRIS) image repository [35]. This database is the first iris biometric database registered in two distinct sessions of VIS and NIR images.

The database has 389 images obtained from 79 individuals for both right and left eyes (326 male images and 63 female images. This database is available to other researchers 111https://utiris.wordpress.com/2014/03/04/university-of-tehran-iris-image-repository/. The people are enumerated in the same way as in NIR and VIS sessions. This database was obtained with an ISG Lightwise LW camera, and the image dimension is 1000 x 776 pixels. An example of database is shown in Figure 3c.

The fourth database was the CROSS-EYED, Reading Cross-spectrum Iris/Periocular Database [36]. This is the benchmark dataset for the identification competition presented in BTAS 2016. This database is available to other researchers 222https://sites.google.com/site/crossspectrumcompetition/cross-eyed-2016. It has eye images captured at the same time using NIR and Visible (RGB) VIS sensors. The images were acquired from a distance of approximately 1.5 m. The NIR wavelength has a single channel of information, while the VIS iris images contain three channels. The images present an indoor environment with realistic illumination conditions. They also include large variations in ethnicity and eye color as well as realistic and challenging illumination reflections. The database has 240 subjects. There are two folders for each subject: one for the NIR and the other one for VIS images. For each folder, there are eight images. In total, we have 632 male images and 328 female images (1,920 images). An example of the database is illustrated in Figure 3d.

IV-A UNAB-Gender Database

As an additional contribution to this paper, we created a database in order to facilitate research in this area. A database at Universidad Andres Bello called UNAB-Gender was created. These images provide to researchers a person-disjoint set for evaluating the gender-classification problem.

This database was also to test and compare our approach with other databases. 333The UNAB-Gender is available to download in https://jtapiafarias.wordpress.com/biometric/database/. The images were captured using the iCAM TD-100 NIR sensor. The iCAM TD-100 uses near-infrared illumination and acquires 480x640 8-bit pixels per image. This set of iris images was obtained over five sessions with 69 female and 66 male subjects. In total, we obtained 1,412 female images and 1,356 male images. All the images are labeled with the gender information.

It is important to note that all the dataset samples from all source were represented in a balanced manner in both training and testing datasets.

V Experiments and Results

In this section, we describe five experiments and seven tests designed to test the performance of the gender classification when using the iris image. An SVM and nine ensemble classifiers as shown in Tables II, and III were used.

The experiments were designed to try different features extracted from the datasets. The tests, on the other hand, were designed to test subsets taken from the databases described in section IV.

For training purpose, we used the UND periocular dataset because has a bigger number of disjoint images. A training portion of 952 and 946 periocular UND images were used for the left and right eyes, respectively.

All the datasets were selected using a person-disjoint method, 60% of the males and 60% of the females images but taking into account a k-fold cross-validation function that ensures the images are unique in each dataset and also unique when considering both datasets together. Therefore, images in the testing set are not present in the validation set.

Once the parameter selection was finalized, the selected parameterization of the method was trained on the full 60% of training data, and we performed a single evaluation on 40% of the test data. In each experiment, an SVM classifier with a Gaussian kernel was trained using LIBSVM implementation [42]. To validate the results, we used NIR-UTIRIS, NIR-Cross-Eyed 2016 and NIR-UNAB-Gender databases to analyze how well the classifier generalizes the new data.

V-A Experiments

We performed five experiments using different inputs for the SVM classifier.

Experiment 1

: In this experiment, we use the raw pixel intensity as the input of the SVM classifier.

Experiment 2

: In this case, the inputs to the SVM classifier are histograms from ULBP (8,1) up to ULBP (8,8). Where each histogram was build considering 59 bins.

Experiment 3

: In this experiment, the inputs to the SVM classifier are a concatenation of ULBP feature extracted from each image (from ULBP (8,1) to ULBP (8,8) in the radii ULBP database) The resulting length is 472 ($59features\times 8radii=472bins$) for each periocular image.

Experiment 4

: For this experiment, the inputs to the SVM classifier are HOG and IHOG features obtained using block sizes of $3\times 3$, $5\times 5$, and $10\times 10$, respectively. In the last three experiments (from 2 to 4) we use a histogram of features. That allows us to reduce the number of features used in the SVM classifier. This reduces considerably the timing of the process which is a factor in most real-time applications.

Experiment 5

: For Experiment (5), we concatenated the information from the histogram of intensity, histogram of texture and histogram of shape (Pixel Values + ULBP(8,1 to 8,8) + HOG ($3\times 3$, $5\times 5$, $10\times 10$)).

V-B Tests for validation

We defined seven tests for validating the correct classification rate. For each test, we use a subset taken from the databases described previously.

Test 1

: In this test, we have considered the correct classification rate using the full UND periocular NIR images (Non occluded), as illustrated in Figure 3a.

Test 2

: In this test, we used the iris artificially masked (occluded condition) from UND periocular NIR images, as illustrated in Figure 3b. The segmentation stage automatically finds the iris and the pupil. It also masks in black the highlight, occlusion and the iris from the periocular image. The normalization, encoding, and comparison stages are not needed in this approach.

Test 3

: In this test, we added a new dataset named UTIRIS. The previous two tests, this dataset was not part of the training set. Therefore, the results allow us to understand how well the previous classifier (Test 1 and Test 2) generalized to the new data. In this test, we considered the correct classification rate using an SVM model that was trained with the UND periocular database (Non-occluded and Occluded) (see Figure 3c).

Test 4

: For additional validation, we use the Cross-Eyed 2016 database. This dataset, like the previous one, was not part of the training set. In this test, we considered the correct classification rate using an SVM model trained with the UND periocular database (Non-occluded and Occluded) (See Figure 3d).

Test 5

: In this test, we use the new database created for this paper, the UNAB-Gender NIR images (See Figure 3c). The database was not part of the training set.

Test 6

: In the previous test (from 1 to 5), we have used the UND Periocular database as a training set. In this case, in particular, we used images from the following databases: UND Periocular+UTIRIS+Cross-Eyed. This combined database was used for training and testing. The use of three features (Intensity, shape, and texture) combined at different scales performed better than using only one feature on a single scale.

In all the experiments, we estimated the correct classification rate separately for the left and right eyes. Since some of the capture systems worked with a single periocular image. For systems capturing both irises, we can use either of them to validate the identification.

The results for the correct classification rate obtained for Tests 1 to 6 are reported in Table IV, TableV, TableVI, and TableVII, respectively. In these tables we report results when using SVM classifier.

Note that we are using different databases where the images may not present exactly the same conditions. Even images that were acquired at the same time (i.e left and right eye) may present different illumination conditions due to sensor position. Specular highlights may also be present in different parts of the image. The inclination of the head, the eyelid occlusion, and the type of camera used may also modify the image.

VI Discussion

In this section we present the results for the six tests and five experiments (section VI-A). In Section VI-B we discuss the most relevant information for gender classification when using periocular images. A statistical ANOVA test to validate our results is shown in section VI-C.

VI-A Gender Classifications results

Table II shows the correct classification rate achieved from periocular images (Non occluded and Occluded) when the images are trained and tested using UND Periocular database (test 1 and 2, experiments 1 to 5). The periocular UND database presents a similar balance between male and female images. For the Non occluded periocular images of the left eyes, the best result was achieved by the fusion of HOG $5\times 5$ with 82.83 +/- 0.5%. For the right eyes, the best result was achieved by the concatenation of HOG $10\times 10$ with 85.74 +/- 0.4%. For the occluded or masked periocular images of the left eyes, the best result was HOG $10\times 10$ with 80.00 +/- 0.5%. For the right eyes, the best result was also achieved by the concatenation of ULBP(8,1) to (8,8) with 84.33 +/- 0.6%.

Based on these results, we can state that there is no significant influence of iris information on correct classification rate when using periocular images. We observed a variation of approximately 4% between the best results when using Non occluded and Occluded images. Table III, also demonstrate this finding when using the nine different classifiers applied to the best experiments and tests from Table II.

In order to show that the results are not dependent the SVM classifier we included new experiments using other classifiers applied to the best results of Table II. We used nine ’ensemble classifiers’ with the periocular UND images: AdaboostM1, LogitBoost, GentleBoost, RobustBoost, LPBoost, TotalBoost, RUSBoost with ’Tree learners’ classifiers and learning rate of 0.1, Random Forest classifier (RF) with 900 ’Trees’. The results are presented in Table III.

Table IV (test 3, experiments 1 to 5), Table V (test 3, experiments 1 to 5), Table VI (test 4, experiments 1 to 5), and Table VII (test 5, experiments 1 to 5), also show the correct classification rate. The results on the Table indicate the performance of the classifier.

In Table VII we show the results for test 6 with experiments from 1 to 5. This test shows results when the training and testing datasets are obtained by combining the Periocular UND, Periocular UTIRIS, and Periocular Cross-Eyed databases. The resulting training dataset has 1,265 images and the Test set has 1,036 images. The results of the correct classification rate are presented in Table VII. The ULBP 8,1 to 8,8 feature extraction achieved the best results for the left and right eyes. This shows that when combining the texture features with different scales (radii) provides better results than using only one scale.

When a dataset is unbalanced (i.e the number of samples in one class is higher than in the other class - which happens with UTIRIS and Cross-Eyed 2016 datasets), the evaluated correct classification rate of a classifier is not representative of the true performance of the classifier. For binary classification, such as problems like gender classification, we can derive two indicators: True Positive Rate(TPR) and True Negative Rate (TNR). Those indicators are commonly used to evaluate binary classifiers. TPR can be expressed as $TP/(TP+FN)$, and TNR as $TN/(TN+FP)$.

In general, the True Negative Rate refers to the correct classification rate on the class negative (male), and the True Positive Rate refers to the correct classification rate on the class positive (female). We can conclude that the male prediction is more robust. This assumption is based on the high value of TPR and the low level of TNR.

Also in Table VIII the Matthews Correlation Coefficient (MCC) [43] of the best results reached for all of the experiments from Table II to Table VII. MCC takes into account true and false positives and negatives. It is a balanced measure which can be used even if the classes are of very different sizes. The MCC is a correlation coefficient between the observed and predicted binary classifications. MCC values returns a value between $-1$ and $+1$. A coefficient of $+1$ represents a perfect prediction, 0 no better than random prediction and $-1$ indicates total disagreement between prediction and observation.

VI-B Feature Relevance

In this paper, we achieved a correct classification rate for the left and the right eyes of 82.83% and 85.74% with Non-occluded images and 80.00% and 84.33% with Occluded images. The best result obtained when using a combination of features was 83.60%. When we merged all of the features extracted (raw pixels, shape, texture) with the same scale did not improve the results using texture or shape alone. Therefore, we estimated the histogram from pixels and concatenated with the histograms of ULBPs and HOGs to improve the results and reduces the length of feature vectors. Overall, the raw pixel produced the worst results.

As we have shown, the iris information is not relevant to classify gender when we used periocular images. Therefore, the most relevant feature for the classification should be located on the surrounded area of the iris (not in the iris). In order to demonstrate our statement, we estimate the relevance of each feature using the Gini Index with the XgBoost algorithm [44]. We tested several threshold values until we achieved the best results which were the correct classification rate of 89.22% when using 4,900 features. This is shown in Table IX.

The relevance of the features was calculated explicitly for each attribute in the dataset. The attributes are ranked and compared to each other. In order to estimate the group of most relevant features, we use a decision tree. In the decision tree, we split the groups by using the relevance information weighted by the number of observations in the node. This differ from traditional decision tree methods where the threshold is used instead of relevance.

The performance was evaluated using the purity measure (Gini index). We select the split points to compute the relevance of the tree by averaging across it. See Table IX.

In Figure 4 we show an example of an image where the most relevant features located with the XgBoost algorithm are highlighted. This image was randomly selected from the periocular UND database. Each image has 19,200 pixels ($160\times 120$ ) and shows the increment of the highlighted features from 500 to $10,000$. If we consider more features (pixels) than necessary, the redundancy of the information increase as shown in the figure 4 (d) with 10,000 features.

Our initial experiments indicated that the relevant features for gender classification from periocular images are spread throughout the whole periocular area with exception of the iris. We validated our hypothesis that iris information contained in periocular images is less relevant than the information outside the iris for gender classification. The NIR images add extra information compared to VIS images. Thus, we can remove this area to improve the image-processing stage and gender classification. There are clear computational advantages to predicting gender using the same images like those in iris-recognition systems without needing other texture representations.

VI-C Statistical Test

We employed an ANOVA test for functional data [45] to analyze the results of Tables II, IV, V and VII, and to confirm that the differences in gender-classification rates of the predictive models, between Non occluded and Occluded periocular images, are not statistically significant. To get this, for each database we performed the experiment five times through a 5-fold cross-validation, and the average correct classification rate results were submitted to the ANOVA test. We applied this method to the results for different periocular images, from the left and right eyes, and also for different features extracted, from the Non occluded and Occluded iris images.

One of the advantages of the ANOVA procedure for functional data is that it allows us to deal with the multiplicity problem, which arises when a large number of statistical test are performed simultaneously. In this case, if decisions about the individual hypotheses are based on the unadjusted marginal p-values, then there is usually a high probability that some of the true null hypotheses will be rejected.

For each experiment, the correct classification rates of different extracted features are modeled as functions, which are compared through the norm:

[TABLE]

where $L$ is the number of features considered and $x_{i}$ is the associated correct classification rate.

The p-values of Cuevas et al. [45] procedure are calculated in terms of the bootstrap simulations where the null hypothesis is $H_{0}$: the mean precision functions between Non occluded and Occluded images are equal”. A p-value $p\leq 0.05$ indicates strong evidence against the null hypothesis, so we can reject the null hypothesis. A p-value $p>0.05$ shows weak evidence against the null hypothesis, so we fail to reject the null hypothesis. Since the p-values $0.1789$, $0.2146$ and $0.1675$ are not lower than the significance level $\alpha=5\%$, then there is no statistical evidence that there exists a difference in the mean correct classification rate obtained in the gender-classification using Non occluded and Occluded iris images. See Figure 5, 6 and 7 in appendix X.

VII Conclusions

In this paper we presented competitive gender-classification results with respect to the state of the art. Even when using several databases captured in different conditions. Our approach does not need the image to be centered by the pupil.

We showed that periocular features have high quality information for classifying gender. The results obtained for classification are similar to other approaches even-though we use fewer features. Most traditional methods use several stages such as acquisition, segmentation, normalization and encoding information while we only use the acquisition stage. We also show the high redundancy (not useful information) present in the images. Therefore, it is possible to reduce the number of features (pixels) using the XgBoost algorithm and to select the most relevance features.

The highest correct classification rate reported in previous works was 85.7% [28]. Also, Kuehlkamp et al. approach [32], complement our result and to validate our conclusion with the analysis of the automatic feature extraction method using deep learning methods. These results confirm that periocular information reaches better results.

In this paper, we achieve a correct classification rate of 89.22% when using the Xgboost algorithm with only 4,900 features and a threshold of 0.00607 for the right eye. However, our results were tested on databases that were not used for training the best parameters. This shows the robustness of our method. Our approach is fully automatic and does not require initialization as state of the art techniques. We use a circular segmentation mask instead of the square one used in [28]. This allows us to better represent the iris shape.

In order to evaluate the results, we included two new databases: Periocular UND database (a subset manually labeled for gender-classification and automatically segmented and masked) and UNAB-Gender a homemade database. Furthermore, we presented results that showed comparable correct classification rate for NIR disjointed with gender-from-periocular databases using the same images as those used in iris-recognition systems. This contribution suggests that Non occluded and Occluded information are very similar, and therefore the NIR information obtained from the iris does not improve gender classification. Gender can be classified with periocular NIR images without the need to include iris information.

The cross-performance analysis using different datasets with different image conditions and captured with different sensors, shows that shape and texture information is more relevant than intensity information. Therefore, we can reduce the amount of information required by the classifiers (SVM and ensemble methods). The concatenation of the texture feature at different scales (instead of using only one) from the same image such as ULBP improves the gender classification.

The XgBoost algorithm was used to select the most relevant features of the periocular image. This allows us to significantly reduce the number of features selected by only using the most relevant ones. This improves the gender-classification rates.

VIII Future work

The iris information is an unique pattern have been shown to be stable over the years. This still can be not established for periocular images because of using some skin information areas. The skin changes with years and some wrinkles appear. Therefore, much more work must be done before establishing a general conclusion even create a new dataset to be able to measure these changes and the validation of the results. This work going in that direction.

IX Acknowledgements

This work was partially supported by the Universidad Tecnologica de Chile - INACAP, Universidad Andres Bello - DCI, and FONDECYT INICIACION N11170189.

X Appendix