CASIA-Iris-Africa: A Large-scale African Iris Image Database

TL;DR

The paper introduces CASIA-Iris-Africa, a large-scale African iris image database designed to address racial bias in iris recognition algorithms, providing a valuable resource for demographically sensitive biometric research.

Contribution

It presents a new extensive African iris database with demographic attributes and baseline performance results, highlighting racial biases in existing iris recognition algorithms.

Findings

Baseline algorithms perform poorly on the new database.

The database reveals racial biases in iris recognition.

Provides a resource for bias mitigation studies.

Abstract

Iris biometrics is a phenotypic biometric trait that has proven to be agnostic to human natural physiological changes. Research on iris biometrics has progressed tremendously, partly due to publicly available iris databases. Various databases have been available to researchers that address pressing iris biometric challenges such as constraint, mobile, multispectral, synthetics, long-distance, contact lenses, liveness detection, etc. However, these databases mostly contain subjects of Caucasian and Asian docents with very few Africans. Despite many investigative studies on racial bias in face biometrics, very few studies on iris biometrics have been published, mainly due to the lack of racially diverse large-scale databases containing sufficient iris samples of Africans in the public domain. Furthermore, most of these databases contain a relatively small number of subjects and labelled images. This paper proposes a large-scale African database named CASIA-Iris-Africa that can be used as a complementary database for the iris recognition community to mediate the effect of racial biases on Africans. The database contains 28,717 images of 1023 African subjects (2046 iris classes) with age, gender, and ethnicity attributes that can be useful in demographically sensitive studies of Africans. Sets of specific application protocols are incorporated with the database to ensure the database's variability and scalability. Performance results of some open-source SOTA algorithms on the database are presented, which will serve as baseline performances. The relatively poor performances of the baseline algorithms on the proposed database despite better performance on other databases prove that racial biases exist in these iris recognition algorithms. The database will be made available on our website: http://www.idealtest.org.

Tables

Table 1. Table 1: Some popular Iris biometrics databases and their racial distribution.

Database Description	Race
ND-IRIS-0405 bowyer2016nd : Captured using the LG2200 sensor. Some of the images in the database correspond to subjects wearing contact lenses, leading to visible image artifacts. Furthermore, the database has a roughly balanced female-male ratio. Subjects: 712; Classes: 56; Images: 64,980 (NIR)	AS (23%); C (70.3%); O (6.7%)
UBIRIS-v1 proencca2005ubiris : Captured using Nikon E5700 comprises images with various degrees of reflections, contrast, luminosity, and focus problems. Subjects: 482; Classes: 241; Images: 1877( VIS)	-
UBIRIS-v2 UBIRISv2 : Captured using Canon EOS 5D DSLR sensor and contains multisession iris images that singularly contain data captured in the visible wavelength, at-a-distance (between four and eight meters), and on the move. Subjects: 522; Classes: 261; Images: 11,102 (VIS)	AS (2%); C (90%); A (8%)
IIT Delhi (IITD-V1) KUMAR20101016 IITDWebsite : Captured using IRIS, JPC1000, and digital CMOS camera from the students and staff at IIT Delhi, New Delhi, India. Subjects: 224; Images: 1120 (NIR)	AS-S (100%)
ICE 2005 phillips2008iris : Captured using LG EOU 2200 and comprises images with a broader range of quality than the sensor would normally acquire, including low-quality images that the sensor will otherwise reject. Subjects: 244; Classes: 112; Images: 2953 (NIR)	-
MMU-v1 mmu_db : From the Multimedia University Iris Database, captured by the LG IrisAccess sensor. The Iris images introduce eyelash obstruction and eye rotation. Subjects: 90; Classes: 45; Images: 450(NIR)	-
MMU-v2 mmu_db : Captured by Panasonic BM-ET100US Authenticam sensor. This is another version of MMU-v1 introduced after one year. Subjects: 200; Classes: 100; Images: 995(NIR)	-
MICHE-I de2015mobile : Captured by the volunteers using their mobile devices: iPhone5, Galaxy Samsung IV (GS4), and Galaxy Tablet II (GT2) on their hands. No restriction was imposed on subjects as they chose any face accessory as they would in real practical application. Subjects: 184; Classes: 92; Images: 3732(VIS)	C (100%)
CROSS-EYED -crossEyed7736915 : This Dual spectrum database contains iris and periocular images captured at a distance of 1.5 meters using a custom multispectral iris sensor under an uncontrolled realistic indoor environment and challenging illumination reflection. Subjects: 240; Classes: 120; Images: 11,520(NIR/VIS)	C (75%); AS (17%); AS-S (4%); A (3%)
Aging Analysis aging6239214 : Private dataset that was captured by LG 400 from 2008 through 2011 and used for a comprehensive aging analysis in iris biometrics. Subjects: 644; Classes: 322; Images: 22,156(NIR)	C (76%); AS (12%); O (12%);
CASIA IrisV1 Ma02irisrecognition : Captured with a custom digital optical sensor over two sessions one month apart. Subjects: 50; Classes: 25; Images: 500 (NIR)	AS (100%)
CASIA IrisV2 casia_v2 : Iris images were captured using two hand-held iris devices (OKI IRISPASS-h and CASIA-IrisCamV2) in an indoor setup over a single session. Subjects: 60; Images: 2400 (NIR)	AS (100%)
CASIA IrisV3-Lamp casia_v3 : Captured using the handheld OKI IRISPASS-h iris device and a Lamp light source turned on/off close to the subject to introduce more intraclass variations. As such, the captured irises have nonlinear deformation due to variations in visible illumination. Subjects: 819; Classes: 411; Images: 16,212 (NIR)	A (100%)
CASIA IrisV4-Thousand casia_v4 : Captured using IKEMB-100 camera from IrisKing. The database contains iris images with eyeglasses and specular reflections and provides a wide range of intraclass variations. Subjects: 2000; Classes: 1000; Images: 20, 000 (NIR)	AS (100%)
CASIA-Iris-Africa : Our proposed database is captured using an IKUSBE30 iris sensor with varying degrees of iris sizes across inter and intraclasses. Subjects: 2046; Classes: 1023; Images: 28,717 (NIR)	A (100%)

Table 2. Table 2: Database Capturing Locations

Location	Notes
Ahmadu Bello University Zaria, Kaduna state.	A diverse population that comes from locations all over Nigeria across multiple ethnicities
Kano State Polytechnic, Kano state.	Diverse age demographics that could not be found elsewhere.
Gandun Albasa street, Kano state.	Availability of specific subjects volunteers that were very difficult to be convinced.
Dabai, Danja, Katsina state.	Attract more diverse subjects of all demographics
Brigate street, Kano state.	Specifically selected because of the availability of subjects of various ethicalities that could not be found elsewhere.
Sabon Gari, Kano state.	Additionally, chosen because of population diversity
Hotoro, Kano state.	Additionally, chosen because of population diversity

Table 3. Table 3: Summary of the generated database

Number of Subjects	1023
Male-Female Ratio	56%-44%
Number of Classes	2046
Number of Images	28717
Resolution of Images	$1088\times 640$

Table 4. Table 4: Iris Segmentation and Localization Results-Category 1

Algorithms	Localization
	Hausdorff	Dice
USIT-WahetUSIT3	0.220	0.293
TVMiriszhao2015accurate	0.033	0.842
OSIRISothman2016osiris	0.027	0.848
	Segmentation
	E1(%)	E2(%)
MASK-RCNNahmad2018unconstrained	2.289	1.145
TVMiris	2.148	1.074
OSIRISothman2016osiris	1.687	0.844
USIT-CahtUSIT3	5.442	2.722
USIT-CahtvisUSIT3	5.566	2.783
USIT-IfppUSIT3	5.429	2.715

Table 5. Table 5: Iris Segmentation and Localization Results-Category 2

Algorithms	Localization
	Hausdorff			Dice
	Mod-1	Mod-2	Mod-3	Mod-1	Mod-2	Mod-3
NIR-ISL2021030902	0.007	0.011	0.007	0.971	0.952	0.971
NIR-ISL2021041402_2	0.023	0.018	0.019	0.853	0.920	0.912
	Segmentation
	E1(%)			E2(%)
	Mod-1	Mod-2	Mod-3	Mod-1	Mod-2	Mod-3
NIR-ISL2021030902	0.299	0.390	0.299	0.149	0.195	0.149
NIR-ISL2021041402_2	2.960	0.909	1.629	1.480	0.455	0.815

Table 6. Table 6: TE-R-V- protocols results: FNMR(%) at various values of FMR

Algorithm	Ideal Set				TE-R-V-FF				TE-R-V-PP				TE-R-V-PF
	EER	10%	1%	0.01%	EER	10%	1%	0.01%	EER	10%	1%	0.01%	EER	10%	1%	0.01%
USIT-Lg	5.21	4.42	7.88	14.35	12.87	14.07	22.88	32.13	22.60	30.63	45.78	58.82	22.32	30.23	45.13	57.61
USIT-Cg	14.67	18.55	45.50	68.85	23.98	39.16	65.52	80.01	37.95	65.82	85.31	93.04	38.75	69.71	89.67	95.48
USIT-Qsw	13.59	15.33	27.96	42.79	23.24	31.21	46.84	58.44	35.51	54.16	70.58	80.27	34.79	52.94	69.44	79.11
USIT-Ko	16.56	21.06	45.85	68.82	25.69	38.12	64.86	82.34	35.36	58.66	81.46	91.22	37.16	63.69	85.34	94.49
USIT-Cr	9.21	8.94	16.45	27.37	17.66	21.91	36.36	49.51	31.37	48.67	67.92	78.94	31.17	48.92	69.20	80.92
USIT-Cb	24.70	35.39	58.12	77.04	30.13	47.25	71.00	84.94	38.78	64.55	84.02	92.12	40.84	70.35	90.08	95.78
USIT-Dct	32.27	70.29	91.37	96.74	37.10	67.14	90.50	96.54	49.04	86.85	96.54	97.83	47.48	86.56	97.50	98.92
UniNet	0.90	0.27	0.85	24.05	10.37	10.57	18.35	24.86	23.87	34.93	60.71	85.91	21.23	29.44	46.20	59.94
Graphnet	14.44	17.24	36.66	58.00	23.02	33.83	57.68	75.44	31.40	53.39	77.40	87.92	34.31	60.12	83.46	92.62
LightCNN	26.51	38.31	60.44	76.17	32.28	50.02	72.56	85.10	34.71	58.88	82.33	91.04	41.30	75.00	91.83	96.59
Maxout	14.48	17.48	32.61	48.90	21.85	30.94	51.72	67.32	31.70	50.67	71.81	82.54	34.31	57.05	78.89	88.61
Afnet	28.83	49.81	77.78	90.89	29.92	51.65	78.60	90.13	38.74	68.04	88.11	94.20	38.86	70.76	91.28	96.91

Table 7. Table 7: Identification results in (%) for protocols TE-R-I-

Algorithm	Ideal Set (%)			TE-R-I-FF (%)			TE-R-I-PP (%)			TE-R-I-FP (%)			TE-R-I-PF (%)
	R-1	R-5	R-10	R-1	R-5	R-10	R-1	R-5	R-10	R-1	R-5	R-10	R-1	R-5	R-10
USIT-Lg	61.99	81.07	83.26	42.27	64.53	67.24	29.45	45.08	48.89	25.45	35.65	38.31	28.97	41.74	44.42
USIT-Cg	39.41	56.31	61.60	17.96	30.74	35.23	9.07	15.34	17.70	5.84	10.16	12.75	7.31	12.88	15.28
USIT-Qsw	48.83	64.49	68.38	29.71	46.49	49.62	19.16	28.51	31.04	16.60	24.27	26.80	19.03	28.62	31.29
USIT-Ko	35.28	52.73	57.63	18.94	31.40	35.30	10.69	18.06	20.74	7.29	11.81	14.38	9.72	16.27	19.19
USIT-Cr	54.28	72.74	75.55	32.41	49.04	52.94	20.44	31.57	35.35	15.96	23.11	26.19	19.05	28.66	31.37
USIT-Cb	26.40	41.12	45.79	11.52	19.47	23.23	7.86	13.74	16.00	4.30	7.86	9.70	5.69	9.53	11.94
USIT-Dct	5.61	8.64	15.73	3.02	4.69	6.37	2.76	4.04	4.63	1.33	1.73	2.25	1.55	2.48	3.21
UniNet	75.77	98.28	98.50	53.61	76.76	78.82	28.88	44.09	48.17	31.13	44.88	47.78	32.89	46.95	49.90
Graphnet	38.71	56.70	61.29	19.94	33.28	37.53	12.39	20.22	23.56	8.11	12.77	15.26	10.66	17.63	20.91
LightCNN	27.02	39.95	46.11	14.37	22.22	25.49	9.52	15.40	18.31	4.90	7.62	9.50	6.26	9.89	11.75
Maxout	44.86	61.60	65.97	25.93	40.59	44.91	15.89	26.85	30.30	11.10	17.55	20.12	13.31	21.54	24.99
Afnet	16.12	30.69	36.92	7.52	14.54	18.69	5.80	11.69	14.60	3.38	6.85	8.80	3.90	7.33	9.34

Table 8. Table 8: Sex demographic analysis in (%) for the protocols TE-R-I-

Algorithm	TE-Ideal-Set (%)				TE-R-I-FF (%)				TE-R-I-PP (%)				TE-R-I-PF (%)
	Rank 1		Rank 5		Rank 1		Rank 5		Rank 1		Rank 5		Rank 1		Rank 5
	M	FM	M	FM	M	FM	M	FM	M	FM	M	FM	M	FM	M	FM
USIT-Lg	64.09	83.38	70.65	82.26	45.63	64.98	50.15	69.69	31.53	48.11	33.97	48.80	31.66	41.80	37.86	48.73
USIT-Cg	42.21	60.37	43.87	61.29	19.90	32.33	24.17	38.76	10.06	16.73	11.96	18.75	7.94	13.23	10.65	18.90
USIT-Qsw	50.67	67.60	53.23	67.42	32.68	47.24	37.12	53.26	20.45	31.21	22.17	30.27	20.94	29.20	26.53	34.28
USIT-Ko	37.46	56.35	43.55	55.48	19.36	30.42	28.42	42.88	11.34	19.37	14.35	22.77	10.92	17.47	12.76	21.58
USIT-Cr	56.86	76.26	60.65	74.52	33.73	49.08	42.54	58.43	22.33	34.37	23.97	34.40	20.58	28.96	26.68	34.82
USIT-Cb	29.21	44.07	33.87	49.03	12.04	20.41	16.86	27.20	8.73	15.78	9.46	15.76	6.27	10.61	7.03	13.59
USIT-Dct	5.99	9.80	9.03	23.87	3.50	5.75	3.15	7.52	3.19	4.70	2.88	4.35	1.58	2.50	2.58	4.20
UniNet	81.66	98.64	85.82	97.52	56.13	78.12	63.04	79.23	32.49	47.82	33.42	47.16	35.34	49.77	37.76	48.23
Graphnet	36.47	57.43	44.89	64.72	20.91	36.11	22.22	37.96	12.47	20.96	15.77	26.29	10.15	17.93	15.05	24.60
LightCNN	26.43	41.17	30.90	48.43	14.95	24.28	16.27	25.00	9.74	16.33	11.76	19.83	4.98	9.13	10.85	16.64
Maxout	43.96	62.90	48.23	69.10	28.05	43.94	26.15	43.92	16.59	27.49	18.84	33.16	12.92	22.01	19.43	29.78
Afnet	17.79	34.56	20.46	37.58	8.75	17.56	9.39	19.40	6.38	12.94	7.81	16.50	3.43	7.31	7.27	12.63

Table 9. Table 9: Age Demographic Analysis in (%) for the protocols TE-R-I-

Algorithm	Rank 1 Identification (%)
	TE-R-I-FP						TE-R-I-FF
	less 20	20-29	30-39	40-49	50-59	greater 60	less 20	20-29	30-39	40-49	50-59	greater 60
USIT-Lg	61.77	48.78	59.12	59.38	54.35	56.28	48.50	37.12	42.45	37.48	40.33	41.85
USIT-Cg	33.39	21.50	31.75	27.50	25.23	32.79	13.17	11.03	10.71	12.58	12.42	25.79
USIT-Qsw	46.99	36.74	43.96	42.81	38.74	38.14	33.65	26.28	29.06	23.41	29.53	29.93
USIT-Ko	31.98	22.87	30.81	29.37	25.53	36.74	16.90	13.19	15.99	12.58	19.14	22.14
USIT-Cr	46.21	37.38	48.93	45.78	43.24	52.09	31.24	24.61	30.36	22.06	32.99	34.55
USIT-Cb	19.39	14.78	19.43	23.59	18.02	24.42	9.66	8.30	10.55	10.01	13.65	13.38
USIT-Dct	7.90	3.34	8.18	2.03	4.50	12.09	2.56	1.83	2.84	3.38	2.85	6.57
UniNet	62.55	59.60	73.88	66.55	69.12	79.04	35.81	39.82	46.91	31.17	52.98	53.16
Graphnet	25.16	19.99	26.10	28.35	47.32	30.82	13.15	11.17	15.76	13.75	33.07	26.49
LightCNN	15.38	14.62	19.90	22.33	33.75	24.21	10.44	5.95	10.26	10.47	11.20	15.41
Maxout	28.14	27.03	36.26	33.40	51.42	32.39	15.70	14.24	19.23	19.15	37.33	30.54
Afnet	10.68	8.58	14.64	17.09	32.18	19.18	6.84	4.55	7.99	7.36	13.87	10.54

Table 10. Table 10: Database comparisons of baseline identification results

Algorithm	EER(%)	Rank-1	Rank-5	Rank-10
Africa-TE-R-V-FF	10.25	54.96	93.40	94.67
Africa-TE-R-I-FP	22.16	45.12	75.01	78.18
Africa-TE-R-V-PF	17.24	52.39	87.63	90.03
Africa-TE-R-I-PF	11.77	52.65	89.38	91.20
Africa-TE-R-I-PP	22.40	45.07	74.58	77.78
Africa-Ideal Set	1.07	58.03	90.66	91.43
Asia-Distance	0.60	99.67	99.73	99.73
Asia-Lamp	1.85	99.73	99.80	99.83
Asia-ND-IRIS	1.55	99.26	99.56	99.68

Equations

$$Openness$$

Full text

1 Introduction

Iris is a biometric trait with phenotypic characteristics that has proven to be agnostic to human natural physiological changes such as aging 2008Image . Recently, there have been tremendous achievements in the research, development, and deployment of iris biometric systems over a wide range of applications such as in security, access control, surveillance, law enforcement, banking, education, etc. These systems have critically evolved in the past two decades partly due to the availability of public iris databases. omelina2021survey .

Various forms of iris databases had been published that address some specific research problems and are made available to the biometrics research community such as constraint databases casia_v2 , that provide use cases for performing fundamental research on iris variability and repeatability of results performance; mobile iris de2015mobile , VISOB_Dataset , that made possible to simulate iris biometric performance in the fast growing mobile environment; multispectral crossEyed7736915 , Toward2616281 , which provide benchmarks for robust evaluation of iris recognition in a dual spectrum paradigm that can lead to enhanced performance; synthetics WVU4100635 , which provides a large-scale iris database with various degrees of controlled imaging conditions that can be used to robustly evaluate iris recognition algorithms; iris at a distance casia_v4 , UBIRISv2 , which present real-time iris images captured by less user cooperation with challenging conditions like the varying iris size, image reflections, occlusion, blur, low resolution, off-axis, motion, specular reflections, etc that can be used to simulate real-time iris recognition performance; contact lenses contact6776569 , special case iris databases captured with contact lenses that provide the unique opportunity for algorithms to evaluate performances on irises covered by different types of contact lenses; liveness detection liveness8272763 , set of databases that enable the critical examinations of algorithms security against spoofing attacks; Aging aging6239214 , proposed to be specially captured over a long period of a time interval of multiple years which can be used for studying the known iris biometric feature of aging.

However, even though these databases have addressed many of the problems mentioned above, most of them contain subjects of primarily Caucasian and Asian docents with very few Africans and many times zero Africans as outlined in the iris databases summary of Table 1. It is particularly challenging as this has created a research blind spot for African cohorts in these databases. Despite many of the reported investigative studies on racial bias in face biometrics Patrick2019Face , Buolamwini1 , cavazos2020accuracy , very few iris racial bias-related studies have been published, which can be due to the insufficient number of other races databases such as the Africans. Recently, face recognition algorithms have been reported to be biased toward specific demographics. Most prominently, many investigative studies have reported higher false-positive rates in African subject cohorts than in other cohorts. Due to the unavailability of the required quantity of African cohorts in the publicly available iris databases, similar studies would be difficult to replicate on iris biometrics. This paper presents a mainly African dataset (samples shown in Fig. 1) that can be useful in addressing some of these challenges. Specifically:

• •

A large-scale African iris image database named CASIA-Iris-Africa is presented and will be made available to the iris biometric research community. The database is captured from 1,023 subjects and has 2046 unique irises with 28,717 images. It can be used as a complementary dataset to the existing databases and helps spur studies on iris biometrics of African-related racial bias. To our knowledge, this database is the first of its publicly available kind.

• •

Sets of application protocols have been developed that will objectively ensure performance variability and repeatability of iris recognition algorithms on the database. These protocols cover potential database application scenarios in iris identification, verification, and classification.

• •

The performance results of various SOTA algorithms on the proposed database protocols are presented, which can be used as the database baseline for further research and provide preliminary insights into racial bias against Africans in iris biometrics.

The remainder of this paper is organized as follows: In section II, related work in the literature will be discussed. Section III describes the proposed database in detail, and the baseline evaluation performance of the database is provided in section IV. Section V concludes the paper and suggests future work.

2 Related Work

2.1 Iris image databases

In Table 1, a summary of some of the most popular iris databases with their race distribution is presented. It can be observed that there are very few subjects in the African cohort. Although many of these databases did not report their race distribution, it is safe to assume that the unreported subjects are not of African descent, considering the origin of these databases, which are mostly acquired from their local communities. Hence, a large-scale African database such as the proposed CASIA-Iris-Africa is necessary to complement these existing databases.

2.2 Iris segmentation and localization

Usually, iris biometrics is based on a constrained setup, whereby the iris images are captured under stringent configurations, producing qualitative iris images that simplify the iris preprocessing task of localization and segmentation. However, more recently, unconstrained iris biometrics, which require less user cooperation and can operate at larger distances, have become more prevalent. This attracts more user applications due to its relaxed invasiveness but also introduces numerous challenges such as occlusion, blur, low resolution, off-axis, motion, and specular reflectionsli2021robust . Various methods have been proposed to address these challenges, which can be broadly categorized ahmad2018unconstrained into specialized, hybrid, and deep learning methods. Specialized approaches propos the use of prior information (such as iris circular/elliptical shape and dark pupil region) that is specific to iris trait features and can be unambiguously identified in an iris image Daugman2009How ,Tan2013Towards ,Gangwar2016IrisSeg . These methods are fast and do not require training images. They can be effective for loosely unconstrained iris images of high quality, as the handcrafted features would be easily identified. However, these methods would be seriously limited for completely unconstrained iris images as the handcrafted features would be less effective. The hybrid methods proposed an alternative approach that complements specialized methods with machine learning techniques to improve the effectiveness of the ILS Tan2012Unified ,Radman2017Automated ,Proena2010Iris . Coarse segmentation is usually obtained with a general machine learning technique, and specialized method is applied to generate the final segmentation. The machine learning algorithms adopted by a hybrid method depend on the training process with iris images. Hence, hybrid methods employ iris priors and ground truth iris images. These methods can be more accurate but slower than specialized methods. However, since they still heavily rely on iris priors, these methods can also be ineffective in challenging unconstrained iris images. To mitigate the problem of unreliable iris priors, deep learning approaches ahmad2018unconstrained , Sardar2020Iris ,Wang2020Towards , propose solutions based on general-purpose semantic segmentation methods. They are more accurate than the former methods and do not require crafting any prior iris features. However, they usually require large-scale training data and are generally slower. In this work, some of the methods in these three categories have been adopted to generate benchmark results for the proposed database.

2.3 Iris representation

Gabor filter fogel1989gabor based features are among the most popular classic iris representation methods Daugman2009How , masek2003recognition . In Daugman2009How , Daugman proposed the application of a 2D Gabor filter on a normalized iris image, and the resulting responses were binarized to produce IrisCodes. Then the Hamming distance was used to measure the similarity between two IrisCodes. In masek2003recognition , a 1D Gabor filter was used instead of a 2D filter to efficiently generate IrisCodes. In sun2008ordinal , rather than the precise measurement of the image structures of the Gabor filter-based features, qualitative relationships between the iris image regions were instead measured as ordinal features using multilobe differential filters. Another classic approach is the use of discrete cosine transforms(DCT) ahmed1974discrete to generate binary features by analysing the frequency of image blocks in a normalized iris image monro2007dct . In miyazawa2008effective , discrete Fourier transforms(DFT) sundararajan2001discrete were employed instead of the DCT to generate the binary features from the frequency of the image blocks.

Beyond the classical methods of iris representation, deep learning-based approaches for extracting the iris features in an image are becoming more prevalent gangwar2016deepirisnet , zhao2017towards , zhang2018deep . In gangwar2016deepirisnet , a convolutional neural network (CNN) trained with normalized iris images was employed with the output vector of the last fully connected layer considered as the iris features. In zhao2017towards , instead of learning direct iris representation from the last layers, the CNN learned spatially corresponding features by stacking the output of multiple layers and convolving them to produce a single channel feature image which is then binarized to generate the IrisCodes. In zhang2018deep , a method coined as MaxOut involves a weighted fusion of the CNN deep features of an iris and periocular biometric modality into a compact representation that can be used for iris recognition.

3 The Database

3.1 Locations

The iris images were captured in Nigeria, Africa. The capturing exercise was conducted across various locations in multiple sessions over three months. Due to logistical constraints, the capturing exercise was limited to the northern part of Nigeria. However, the locations were strategically selected within the region to improve the subjects’ diversity. These locations are in urban cities, schools, and local neighborhoods. The locations are shown in Table 2. Each subject volunteer was asked to sign a consent form authorizing the data to be used for only research purposes.

3.2 Image capture

The iris sensor device for capturing the images is the IKUSBE30 iris sensor from IrisKing shown in Fig. 2. The setup for the capturing exercise is shown in Fig. 3. Each volunteer was asked to hold the device, standing or sitting, based on their preference. As such, the iris images were captured at various degrees of head postures.

The procedure for image capturing is in two steps:

• •

*Step 1-Straight Gaze Capture: * In this step, the volunteer was asked to hold the sensor and gaze into its lenses while moving the eyes in small ranges to the left, right, up, or down for approximately 3 minutes. During this time, images were periodically captured automatically at constant intervals. The eye movement ensures that the iris position and orientation are highly diversified across the captured images. Samples of the captured images in this step are shown in Fig. 4 (left).

• •

*Step 2- Open-Close Capture: * In this step, the volunteer was asked to open and close the eyes while gazing into the device for approximately 3 minutes. The iris images were automatically captured within this period. The essence of the opening and closing of the eyes is to ensure that the images contain irises of various sizes and degrees of occlusion, from fully closed eyes to half open and fully open eyes. This procedure will ultimately improve image diversity. Exemplary samples are shown in Fig. 4 (right).

All the iris images were acquired using this two steps procedure. As such, for every subject, there were two sets of images captured with a maximum of 84 images per set which means that up to 168 images were captured and saved for each subject.

3.3 Database cleaning and construction

After the capturing sessions, the stored images were processed. The processing steps comprise automatic image selection to discard images with duplicate content, followed by manual image selection that guarantees the selected images’ reliability. In the automatic selection, all the images from a single eye of one subject were organized into a matrix, with each column representing an image. The correlation coefficients of this matrix were then computed based on the technique proposed in teukolsky1992numerical . The values of the computed coefficients range from 0 (no correlation) to 1(full correlation). Based on these coefficients, all pairs of images with coefficient values greater than the threshold $T$ are considered duplicates, and one of them must be discarded. A threshold of $T=0.82$ was heuristically chosen after careful adjustments with multiple values. The images of the subject’s eyes were then processed, and duplicates were removed using this procedure. The remaining images were then manually processed. The manual selection involves one-by-one human inspection of each image to identify damaged images and those images with no irises captured in them. A GUI selection tool shown in Fig. 5 has been specially developed for this task, allowing efficient image selection. The tool provides the means of using just the mouse left click to select an image and right-click to discard an image. The selection decision is recorded on each mouse click, and the next image in the list is immediately loaded and displayed. This makes inspection very fast and can lead to the processing of up to 55 images per minute. All the images were inspected using this tool, and the unselected images were discarded. The remaining images constitute the generated dataset. Some sample images of one subject from the generated dataset are shown in Fig. 7, the summary of the database is presented in Table 3 and the database subjects’ age distribution is shown in Fig. 6.

3.4 Database labelling

After the database has been generated and organized, each iris image needs to be appropriately labelled to be effectively utilized for iris recognition tasks. The labelling process comprises two tasks: (1) pixel-wise labelling of the iris region to generate an iris segmentation mask; and (2) drawing of circular contours around the iris and pupil regions to yield the iris inner and outer cycle parameters. These tasks are tedious and delicate as they entail carefully segregating the boundaries around the iris region from the incursion of eyelids and eyelashes. We adopt a semiautomatic labelling approach. First, 1010 images were carefully selected and manually labelled as discussed in Wang2020Towards . These images were then used to train two CNN-based models performing segmentation and localization. The trained models were then used to label all the remaining images in the database automatically. The GUI tool presented in Fig. 5 was again used to inspect the labelled images manually, and all the observed labelling errors were manually relabelled. Sample images of the labelled database are shown in Fig. 8.

After the labelling, each image has a corresponding segmentation mask and iris inner and outer cycle parameters. Additionally, an openness score is added to provide more contextual information on the database composition. It is based on the labelling information of each iris image. The openness score is an empirical measure of iris visibility. Since the iris images of the same subject were deliberately captured at various degrees of visibility/sizes, as discussed in the previous section, incorporating this information will be important in analyzing methods evaluated with the proposed database. The openness score is computed as the ratio of the number of pixels covered by the mask to the number of pixels covered by the iris outer cycle. As such, an image of a wide-open eye will have an iris with an openness score close to 1, and a partially closed eye will have an iris with an openness score much less than 1. Mathematically, the openness score is the complement of the pixelwise XOR operation between the segmentation mask $M$, and the iris outer boundary masks $O$ (filled cycle of the outer boundary). It can be written as:

[TABLE]

where $n$ corresponds to the number of pixels covered by the iris outer boundary mask in the image. In Fig. 9, the openness scores for some sample images are shown. As expected, it can be observed that the partially closed eye (Fig. 9(a)) has a very low visibility of about 0.07. In contrast, wide-open eyes have very high visibility of about 0.99. The distribution of the openness scores of the images in the generated dataset is shown in Fig. 10. Based on these values, a cut-off threshold of 0.8 was chosen to divide the dataset into iris images with partial (P) openness (score below 0.8) and iris images with full (F) openness (above 0.8). As such, the attribute of either P or F is added to the labelling information of each image in the database.

3.5 Application oriented database protocols

Inspired by muhammad2021casia , a group of protocols is proposed, ensuring consistent results in adopting the proposed database. Unlike the strategy of defining protocols as a list of images by the existing proposed databases, this paper proposes application-oriented protocols that attempt to be closely related to the real-world application of iris biometrics. First, the database subjects are divided equally into training and testing partitions with no overlap of subjects between the two sets. The division is performed so that the subject’s attributes are adequately represented across the two partitions. The protocols for the testing partition are shown in the protocol tree of Fig. 12 and those of the training partition are shwon in Fig. 13. These protocols are fundamentally based on two factors: (1) database application for either recognition (R) or classification (C) tasks and (2) openness of either full (F) or partial (P) as shown in Fig. 12 and Fig. 13.

3.5.1 Recognition protocols

Each recognition protocol has a gallery setting and a query set emulating the real-life scenario of enrolled subjects (gallery set) and search subjects (query set) in an iris recognition application. The three main recognition tasks covered by the proposed protocols are: (1) close-set identification (I); (2) open set identification (O); and (3) verification (V) as shown in Fig. 12 and Fig. 13.

• •

The close-set identification protocols contain one image per subject in the gallery set and multiple images per subject in the query set. A subject must have images in both sets. As such, ranking-based evaluation metrics can be used to evaluate images in this group of protocols.

• •

The open-set identification protocol is similar to the close-set, except that a subject may not have images in the gallery set. This is analogous to the identification of subjects who have not been previously enrolled. A combination of ranking-based evaluation metrics with a predefined threshold can be used to evaluate these groups of protocols.

• •

For the verification protocols, all images of subjects are included in both the gallery and query set such that mated and non-mated pair combinations can be generated from them. Many of these pair combinations with highly similar pairs can lead to redundant results. Therefore, to reduce the redundancy, the pairs were generated by maximizing the intraclass distance and minimizing the inter-class distances. This can be achieved using the absolute difference of the openness score of the images as the measure of pair similarity to choose only the highly dissimilar pairs of the mated pairs and discard the highly dissimilar pairs of the non-mated pairs.

3.5.2 Classification protocols

The classification groups of protocols are based on the three available subject attributes in the database : (1) ethnicity (E); (2) age (A); and (3) sex (S) as shown in Fig. 12 and Fig. 13. Unlike in recognition, the classification protocol has only one set of images that can be used to evaluate the respective classification models. For ethnicity classification, each of the six major ethnicities with a relatively high number of subjects was considered an individual class, while all the remaining ethnicities were categorized as others. This is necessary as some ethnicities have very few subjects which can lead to class unbalancing. All the distinct subject age values are considered separate classes for age classification. This will provide the flexibility of either performing age group classification or age regression. For the sex classification, the classes are the binary male or female classes. All the subjects can be classified to either one of these values.

3.5.3 Iris openness in protocols

In addition to the database application considered for protocols generation, the iris image openness attribute described in the previous section has also been considered. The intuition here is to objectively evaluate algorithms’ sensitivity to the iris’s visibility in the African irises’ context. The pair order permutations of PP, FF, FP, and PF have been considered for recognition protocols that require image pair comparisons. This corresponds to both images in the pair having partial openness attributes (PP), full (FF), or the query image full and gallery image partial (FP), and vice versa in PF. For the classification protocols, the image openness attribute is directly considered as the classification task typically involves the analysis of a single image.

3.5.4 Ideal subset protocols

To provide a simplified set of images that can be quickly evaluated by algorithms, an ideal subset of images is generated by selecting images from the testing set with complete visibility ( openness score greater than 0.89) and a reasonably fixed front elevation (iris orientation $<=0.4$ radians ). This set is adopted in both the recognition and identification protocols. Sample images of the ideal set are shown in Fig. 11.

3.5.5 Final protocols

After considering all the stated factors, the complete set of protocols is shown in Fig. 12 as the tree leaves, which comprises a total of 18 sets of protocols for the testing and 4 sets of protocols for the training set. Prospective users can choose one more of these protocols to adopt the proposed database appropriately.

4 Experiments and Baseline Performances

4.1 Baseline algorithms

To provide benchmark evaluations of the proposed database, state-of-the-art (SOTA) open-source algorithms have been evaluated as baseline algorithms for the proposed database. The results will be presented in the next section. The baseline algorithms for the localization and segmentation task include the popular USIT USIT3 open source iris software package algorithms of USIT-Wahet, USIT-Caht, USIT-Cahtvis, and USIT-Ifpp. Other algorithms are MASK-RCNN ahmad2018unconstrained , TVMiriszhao2015accurate , OSIRISothman2016osiris and 2 of the top 3 best participants algorithms of the NIR-ISL 2021 iris recognition challenge111https://sites.google.com/view/nir-isl2021/home tagged as NIR-ISL2021030902 and NIR-ISL2021041402_2 wang2021nir . The baseline algorithms for the iris recognition task also include some of the USIT implementation algorithms such as USIT-Lg; USIT-Cg; USIT-Qsw; USIT-Ko; USIT-Cr; USIT-Cb; and USIT-Dct. Other algorithms include UniNet zhao2017towards , GraphnetRen2020DynamicGR , LightCNNWu2015ALC , MaxoutZhang2018DeepFF , and AfnetRen2019AlignmentFA .

Many researchers have widely adopted all these algorithms, and their implementations are readily available. As such, it will be particularly appropriate to present the baseline performance of the algorithms on the proposed database.

4.2 Iris segmentation and localization

The baseline results of the localization and segmentation algorithms were evaluated against the ground truth of the constructed dataset. The evaluation criteria used for localization are the Hausdorff distance and Dice metrics. Hausdorff distance measures the shape similarity between the ground truth and predicted inner/outer boundary shapes, while Dice measures the overlap between these shapes. For the segmentation task, E1 and E2 error rates are used wang2021nir . E1 is the average proportion of corresponding disagreeing pixels over all the images. It is calculated by the pixelwise XOR operation between the predicted segmentation mask and its ground truth wang2021nir . At the same time, E2 is the average of false-positive and false-negative rates. The generated results are in two categories: (1) results of USIT and Mask-RCNN algorithms; and (2)results of NIR-ISL 2021 algorithms.

• •

Category 1: the results of the USIT and Mask-RCNN algorithms are shown in Table 4. All the USIT-based algorithms perform poorly, while the deep learning-based Mask-RCNN algorithm has a relatively better error rate.

• •

Category 2: The algorithms from the NIR-ISL 2021 competition each have three models trained on three different datasets. These three models are tagged as Mod-1, Mod-2, and Mod-3, and the results are shown in Table 5. It can be observed that some of these models perform poorly while some exhibit relatively better performance.

4.3 Iris recognition

Before extracting the iris representation from the images used for iris recognition evaluations, the images of the generated database were segmented and localized using the ground truth labelLing data from the database. Each image was first segmented and normalized to a rectangular image of size $64\times 512$ using Daugman’s rubber sheet model Daugman2009How .

4.3.1 Verification results

The results for the verification of the TE-R-V-xx group of protocols are presented in Table 6 and their corresponding detection error tradeoff (DET) curves are shown in Fig. 14. It can be observed that the baseline algorithm performances are generally weak across all the protocols. Almost all the algorithms fail at 0.01% FMR, which is a standing operating point for real-time application. The results of the algorithms are relatively better on the TE-R-V-FF protocol and worse on the TE-R-V-PF protocol. This can be attributed to the fact that the pairs in TE-R-V-FF have images with similar openness scores and hence similar iris visibility between the FF pairs, while in TE-R-V-PF, it is the opposite case. These results demonstrate the inherent challenges in off-the-shelf algorithms on niche use cases such as African irises. Hence, the proposed database’s availability is needed so that researchers can fine-tune their algorithms.

[FIGURE:]

4.3.2 Identification results

The results for the identification protocols of TE-R-I- are presented in Table 7 and their corresponding cumulative matching characteristics (CMC) curves are shown in Fig. 15. The poor performances of the algorithms across the protocols coincide with the outcome of the verification results. The results are slightly better for the TE-R-I-FF protocol than for TE-R-I-FP and TE-R-I-PF due to similar visibility on TE-R-I-FF pair comparisons. However, it is interesting to observe that the results of TE-R-I-PF are generally relatively better than those of TE-R-I-FP. This gives an interesting insight: probing with a partially occluded African iris (P openness) against a less occluded iris (F openness) yields better results. The presented identification results further demonstrate the value of the proposed database as a complementary database of Africans by the iris biometric research community.

[FIGURE:]

4.4 Genuine and impostor distribution

To analyse the performance of the baseline algorithms, the genuine-imposter distribution of the algorithms with their respective Decidability Index (DI)daugman1998recognizing on the protocols TE-R-V-PF, TE-R-V-FF and TE-R-V-PP are shown in Fig. 18, Fig. 17 and Fig. 18 respectively. This distribution can be used to analyze the iris representations from these algorithms. It can be observed that, as evident from the previous results, the features of the TE-R-V-FF protocol extracted by the algorithms show much better genuine-imposter pair separation with a DI of up to 1.8269 on some algorithms. The lowest DIs are all from the TE-R-V-PF protocols. In general, all the protocols have bad imposter-genuine separation by all the algorithms.

4.5 Demographic analysis

To further analyse the effect of the database demographic attributes on the baseline results, the age and sex demographics performance of these algorithms is presented in Table 8 and Table 9 respectively. For the sex demographics, it can be observed that the result for the female cohorts is consistently better than that of the males across all the algorithms and protocols. For some algorithms, the deviation is up to 100%. For the age demographics, it can be observed that the baseline results of the age group $<20$ years is consistently lower than all others across the protocols and algorithms, while that on the $>60$ age groups is consistently better. It is important to note that fewer samples of the$>60$ age group are in the database.

[FIGURE:]

[FIGURE:]

4.6 Comparison with other databases

Observing how the baseline algorithms perform poorly on the proposed database, the same algorithms have been evaluated on the subsets of some of the publicly available datasets. The selected databases are ND-IRIS-0405 (ND-Iris) bowyer2016nd , CASIA IrisV3-Distance (Distance) casia_v3 , and CASIA IrisV3-Lamp (Lamp) casia_v3 . For fair comparisons, the same localization, segmentation, normalization, and feature extraction strategy are used across all the databases. The Uninet model results were adopted for this comparison. The results of the verification DET curve are shown in Fig. 19 and the identification results are shown in Table 10. Despite performing relatively better on most of the other databases, the model performs poorly on the proposed dataset protocols. This further demonstrates the racial biases of these algorithms. Even though the model has better performance on the Ideal set protocol, at higher FNMR, the performance is poor, similar to the other proposed database protocols.

5 Conclusions and Future Work

This paper presents a large-scale African database that can be used to mitigate racial bias in African iris biometrics. The database contains 1,023 subjects with 2046 unique irises and 28,717 images. A database application protocol has been presented to provide the variability and repeatability of results across algorithms. The baseline performances of some SOTA algorithms have also been presented, proving the existence of iris racial bias in some of these algorithms. In future work, a more comprehensive analysis of African iris biometrics will be conducted using the proposed CASIA-Iris-Africa, and algorithms will be developed by utilizing the database to address the racial bias problem of African iris biometrics.

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grants: 62176025, 62071468, 62006225, in part by Strategic Priority Research Program of the Chinese Academy of Sciences under Grant XDA27040700, and the CAS-TWAS President’s Fellowship for International Doctoral Students