Face4FairShifts: A Large Image Benchmark for Fairness and Robust Learning across Visual Domains

TL;DR

Face4FairShifts introduces a large-scale facial image benchmark to evaluate fairness and robustness of models across diverse visual domains, highlighting current limitations and guiding future development of equitable AI systems.

Contribution

The paper presents Face4FairShifts, a comprehensive dataset and benchmark for assessing fairness and domain generalization in facial image analysis.

Findings

Existing datasets have significant limitations for fairness evaluation.

Models show performance gaps under distribution shifts.

The benchmark reveals the need for better fairness-aware domain adaptation methods.

Abstract

Ensuring fairness and robustness in machine learning models remains a challenge, particularly under domain shifts. We present Face4FairShifts, a large-scale facial image benchmark designed to systematically evaluate fairness-aware learning and domain generalization. The dataset includes 100,000 images across four visually distinct domains with 39 annotations within 14 attributes covering demographic and facial features. Through extensive experiments, we analyze model performance under distribution shifts and identify significant gaps. Our findings emphasize the limitations of existing related datasets and the need for more effective fairness-aware domain adaptation techniques. Face4FairShifts provides a comprehensive testbed for advancing equitable and reliable AI systems. The dataset is available online at https://meviuslab.github.io/Face4FairShifts/.

Tables

Table 1. Table 1 : Different Types of Distribution Shifts shao2024supervised ; fedora . Let X , Y , Z X,Y,Z represent variables of data features, class labels, and sensitive attributes, respectively, with superscripts s , t s,t indicating source and target domains.

Type of Shifts	Notations	Descriptions of Shifts
Covariate Shifts shimodaira2000improving	${\mathbb{P}}_X^s\ne {\mathbb{P}}_X^t$	Marginal distribution shifts on $X$
Semantic Shifts wang2003mining	${\mathbb{P}}_Y^s\ne {\mathbb{P}}_Y^t$	Marginal distribution shifts on $Y$
Demographic Shifts giguere2022fairness	${\mathbb{P}}_Z^s\ne {\mathbb{P}}_Z^t$	Marginal distribution shifts on $Z$
Concept Shifts conceptshift	${\mathbb{P}}_{Y|X}^s\ne {\mathbb{P}}_{Y|X}^t$	Conditional distribution shifts on $Y|X$
Correlation Shift roh2023improving	${\mathbb{P}}_{Y,Z}^s\ne {\mathbb{P}}_{Y,Z}^t$	Joint distribution shifts on $(Y,Z)$

Table 2. Table 2 : Overview of Experimental Datasets

Datasets	Domains	Anno.	Tasks
			FairL	OOD Gen.	OOD Det. (Sec. 3.4)			FairOG
			(Sec. 3.2)	(Sec. 3.3)	Sensory	Intra-Sem.	Inter-Sem.	(Sec. 3.5)
CelebA CelebA	-	40	✓	✗	✗	✓	✗	✗
UTKFace utk	-	3	✓	✗	✗	✓	✗	✗
FairFace fairface	7 (Races)	3	✓	✓	✓	✓	✓	✓
UTK-FairFace utk-fairface	2 (Datasets)	3	✓	✓	✓	✓	✓	✓
Face4FairShifts	4 (P,A,C,S)	39	✓	✓	✓	✓	✓	✓

Table 3. Table 3 : Performance of Fairness Learning Using Multiple Baselines.

Label	Dataset	Accuracy							$\Delta \text{DP}$							$\Delta \text{EO}$
Sen.		LFR	GSR	AD	CSAD	FNF	$ϵ$	$\pm$	LFR	GSR	AD	CSAD	FNF	$ϵ$	$\pm$	LFR	GSR	AD	CSAD	FNF	$ϵ$	$\pm$
Age Gender	Celeba	78.92	80.49	78.48	85.37	85.96	0.043	0.036	0.062	0.069	0.067	0.049	0.047	0.012	0.010	0.062	0.051	0.051	0.036	0.036	0.013	0.011
	UTKFace	78.13	78.27	78.93	83.29	83.50	0.032	0.027	0.021	0.020	0.021	0.020	0.019	0.001	0.001	0.063	0.070	0.070	0.053	0.052	0.011	0.009
	FairFace	87.32	83.21	83.87	89.25	89.69	0.037	0.030	0.071	0.025	0.045	0.025	0.026	0.023	0.020	0.083	0.064	0.065	0.059	0.060	0.010	0.010
	UTK-FairFace	86.43	79.38	78.40	84.44	83.87	0.042	0.034	0.045	0.019	0.021	0.018	0.018	0.011	0.011	0.047	0.040	0.042	0.033	0.030	0.009	0.007
	Face4FairShifts	75.06	74.50	75.90	78.49	79.28	0.026	0.021	0.092	0.083	0.083	0.079	0.071	0.009	0.008	0.086	0.083	0.083	0.074	0.070	0.008	0.007
Age Race	UTKFace	76.90	76.37	75.44	78.81	77.24	0.015	0.012	0.096	0.121	0.100	0.083	0.097	0.016	0.014	0.098	0.088	0.089	0.084	0.087	0.006	0.005
	FairFace	87.42	84.23	82.71	88.90	88.89	0.034	0.028	0.144	0.134	0.129	0.126	0.123	0.010	0.008	0.129	0.120	0.123	0.113	0.119	0.007	0.006
	UTK-FairFace	84.67	81.63	81.62	85.60	85.19	0.023	0.020	0.122	0.094	0.098	0.092	0.095	0.013	0.013	0.102	0.086	0.088	0.086	0.087	0.007	0.007
	Face4FairShifts	73.53	73.97	74.70	76.37	75.30	0.014	0.011	0.154	0.150	0.155	0.145	0.143	0.006	0.005	0.142	0.136	0.142	0.136	0.132	0.006	0.005

Table 4. Table 4 : Average Performance of OOD Generalization across Domains.

Label	Dataset	Accuracy								F-1 Score
		ERM	IRM	GDRO	Mixup	MMD	MBDG	$ϵ$	$\pm$	ERM	IRM	GDRO	Mixup	MMD	MBDG	$ϵ$	$\pm$
Age	FairFace	92.35	92.19	88.68	86.38	92.46	94.46	0.035	0.030	63.92	63.03	16.31	2.60	50.66	65.93	0.315	0.274
	UTK-FairFace	90.20	89.32	88.47	81.31	89.43	91.34	0.037	0.036	62.07	61.83	9.32	14.14	58.25	63.93	0.280	0.258
	Face4FairShifts	87.99	88.36	88.21	88.41	88.22	90.23	0.008	0.008	25.71	35.49	3.75	5.95	46.32	41.29	0.219	0.181
Gender	FairFace	87.90	87.71	57.11	49.85	52.78	90.85	0.227	0.196	86.29	86.36	46.38	47.27	14.14	87.80	0.351	0.303
	UTK-FairFace	85.00	83.77	57.37	47.30	47.92	88.33	0.229	0.195	85.10	81.92	44.66	41.12	24.66	83.92	0.312	0.265
	Face4FairShifts	79.00	77.58	55.75	51.09	46.31	85.26	0.200	0.167	81.62	79.85	43.45	41.99	32.25	82.45	0.271	0.234

Table 5. Table 5 : Average Performance of Inter-Domain Sensory OOD Detection across Domains using Age as Class Labels.

Dataset	AUROC							AUPR							InD/OOD Accuracy (Reference Only)
	ocSVM	DDU	MSP	Energy	Entropy	$ϵ$	$\pm$	ocSVM	DDU	MSP	Energy	Entropy	$ϵ$	$\pm$	ocSVM	DDU	MSP	Energy	Entropy	$ϵ$	$\pm$
FairFace	50.13	52.39	51.27	50.83	50.04	0.012	0.010	12.83	22.91	17.63	16.36	13.83	0.053	0.042	50.13	51.77	52.03	50.85	50.01	0.011	0.009
UTK-FairFace	52.29	54.91	53.83	54.57	51.85	0.017	0.014	18.27	24.83	22.77	26.66	15.27	0.059	0.047	51.63	53.59	52.00	53.34	51.36	0.012	0.010
Face4FairShifts	64.29	66.28	66.01	65.53	64.77	0.010	0.008	45.26	51.25	48.13	50.03	47.28	0.029	0.023	60.02	61.68	61.17	60.90	60.31	0.008	0.007

Table 6. Table 6 : Performance of Intra-Domain Semantic OOD Detection using Age as Class Labels.

Dataset	AUROC							AUPR							InD Classification Accuracy (Reference Only)
	ocSVM	DDU	MSP	Energy	Entropy	$ϵ$	$\pm$	ocSVM	DDU	MSP	Energy	Entropy	$ϵ$	$\pm$	ocSVM	DDU	MSP	Energy	Entropy	$ϵ$	$\pm$
FairFace	78.33	95.89	92.40	92.06	84.08	0.087	0.072	60.12	82.15	77.04	76.45	73.08	0.096	0.083	69.40	79.67	77.48	74.01	76.23	0.048	0.039
UTK-FairFace	88.06	84.68	89.63	93.92	87.02	0.042	0.034	64.07	60.28	77.68	84.80	70.32	0.125	0.100	75.00	84.23	69.07	61.03	67.10	0.109	0.088
Face4FairShifts	71.11	74.76	72.73	69.81	70.25	0.025	0.020	61.09	57.72	56.51	66.37	54.90	0.055	0.046	55.45	61.27	57.16	61.59	57.42	0.033	0.027

Table 7. Table 7 : Average Performance of Inter-Domain Semantic OOD Detection across Domains using Age as Class Labels.

Dataset	AUROC							AUPR							InD Classification Accuracy (Reference Only)
	EDst	SCONE	DAML	MEDIC	MADOD	$ϵ$	$\pm$	EDst	SCONE	DAML	MEDIC	MADOD	$ϵ$	$\pm$	EDst	SCONE	DAML	MEDIC	MADOD	$ϵ$	$\pm$
FairFace	70.28	71.28	69.28	71.93	74.25	0.023	0.019	32.48	36.27	41.28	44.27	47.28	0.075	0.060	66.00	62.94	61.76	59.28	61.27	0.030	0.025
UTK-FairFace	75.22	72.22	77.28	80.29	80.14	0.042	0.034	48.27	44.42	49.28	39.20	50.13	0.053	0.045	71.29	72.15	77.28	75.27	75.81	0.031	0.025
Face4FairShifts	55.16	56.13	55.28	57.28	57.18	0.012	0.010	27.18	26.94	25.81	28.92	29.38	0.018	0.015	52.38	52.39	52.59	54.02	53.02	0.008	0.007

Table 8. Table 8 : Average Performance of Fairness-aware OOD Generalization across Domains.

Label	Dataset	Accuracy								$\Delta \text{DP}$								$\Delta \text{EO}$
Sen.		FCR	EIIL	FVAE	FATDM	FLAIR	FEDORA	$ϵ$	$\pm$	FCR	EIIL	FVAE	FATDM	FLAIR	FEDORA	$ϵ$	$\pm$	FCR	EIIL	FVAE	FATDM	FLAIR	FEDORA	$ϵ$	$\pm$
Age Gender	FairFace	87.41	82.12	86.31	91.32	91.73	92.42	0.048	0.040	0.102	0.151	0.132	0.112	0.101	0.105	0.023	0.020	0.133	0.140	0.136	0.122	0.112	0.102	0.018	0.015
	UTK-FairFace	84.21	80.27	83.24	89.12	88.27	90.12	0.047	0.039	0.082	0.120	0.124	0.083	0.092	0.090	0.021	0.019	0.119	0.120	0.143	0.125	0.129	0.102	0.016	0.013
	Face4FairShifts	78.91	76.13	74.62	81.44	82.25	82.22	0.039	0.033	0.142	0.149	0.151	0.133	0.130	0.125	0.013	0.011	0.143	0.152	0.154	0.152	0.148	0.132	0.009	0.008
Age Race	UTK-FairFace	79.08	72.83	78.91	78.29	77.22	78.52	0.024	0.024	0.143	0.103	0.173	0.154	0.143	0.140	0.026	0.023	0.142	0.113	0.173	0.133	0.143	0.133	0.023	0.020
	Face4FairShifts	72.25	69.13	72.82	73.28	74.22	74.53	0.022	0.019	0.173	0.169	0.179	0.163	0.161	0.160	0.009	0.008	0.169	0.153	0.177	0.145	0.151	0.146	0.015	0.013

Table 9. Table 9 : Overview of Related Real-World Image Datasets.

Datasets	Year	Data Sources	Dataset Size	# Annotations	Sensitive	# Domains	Domains
					Attribute
Face Attribute
MORPH [58]	2006	Public Data	55K	7	Yes	-	-
PubFig [42]	2011	Celebrity	13K	73	Yes	-	-
CACD [13]	2015	Celebrity	160K	9	Yes	-	-
LFWA+ [48]	2015	LFW [33]	13K	40	Yes	-	-
CelebA [48]	2015	CelebFace [70, 71], LFW [33]	202K	40	Yes	-	-
IMDB-WIKI [61]	2016	IMDB, WIKI	500K	8	Yes	-	-
FotW [22]	2016	Flickr	25K	10	Yes	-	-
UTKFace [88]	2017	MORPH [58], CACD [13], Web	20K	3	Yes	-	-
PPB [12]	2018	Government Official Profiles	1K	2	Yes	-	-
LFW+ [29]	2018	LFW [33]	15K	3	Yes	-	-
DiF [50]	2019	Flickr	1M	10	Yes	-	-
CelebAMask-HQ [43]	2020	CelebA-HQ [39]	30K	19	Yes	-	-
CelebA-Spoof [87]	2020	CelebA [48]	625K	43	Yes	-	-
CelebA-Dialog [34, 35]	2021	CelebA [48]	30K	45	Yes	-	-
Distribution Shifts
Office [63]	2010	Web, Digital SLR Camera, Webcam	4K	No	No	3	Amazon, Webcam, DSLR
VLCS [74]	2011	VOC2007 [23], LabelMe [75]	10K	No	No	4	Caltech101, LabelMe
		Caltech101 [25], SUN09 [14]					SUN09, VOC2007
PACS [44]	2017	Caltech256 [27], Sketchy[66]	9K	No	No	4	Photo, Art, Cartoon, Sketch
		TU-Berlin[20], Google Images
WaterBird [65]	2019	CUB [78], Places [91]	9K	No	No	2	Correlations Between
							Birds and Backgrounds
DomainNet [56]	2019	Internet	600K	No	No	6	Sketch, Real, Quickdraw
							Painting, Infograph, Clipart
Camelyon17 [41]	2021	WILDS [41]	400K	No	No	5	Medical Centers
FMoW [41]	2021	WILDS [41]	500K	No	No	3	2002–2013, 2013–2016, 2016–2018
						5	Africa, Americas, Oceania, Asia, Europe
YFCC100M-FDG [89]	2024	YFCC100M [73]	90K	1,570	No	3	Before 1999, 2000-2009, 2010-2014
Face Attribute and Distribution Shifts
FairFace [38]	2019	Flickr, Twitter, Newspapers, Internet	108K	3	Yes	7	Black, East Asian, Indian, Latino
							Middle Eastern, Southeast Asian, White
UTK-FairFace [7]	2022	UTKFace [88], FairFace [38]	128K	3	Yes	2	UTKFace [88], FairFace [38]
Face4FairShifts	2025	CelebA [48], MetFaces [40]	100K	39	Yes	4	Photo, Art, Cartoon, Sketch
		WikiART [72], Human-Art [37]
		IIIT-CFW [51], iCartoonFace [90]
		CUFSF [86], FS2K [24], Internet

Table 10. Table 10 : Complete Fairness Learning Results Using Multiple Baselines. (Extended from Table 3 with Standard Deviations in Parentheses)

Label	Dataset	Accuracy
Sen.		LFR	GSR	AD	CSAD	FNF	$ϵ$	$\pm$
Age Gender	Celeba [48]	78.92 (0.007)	80.49 (0.004)	78.48 (0.036)	85.37 (0.009)	85.96 (0.020)	0.043	0.036
	UTKFace [88]	78.13 (0.012)	78.27 (0.003)	78.93 (0.044)	83.29 (0.005)	83.50 (0.012)	0.032	0.027
	FairFace [38]	87.32 (0.003)	83.21 (0.009)	83.87 (0.032)	89.25 (0.003)	89.69 (0.020)	0.037	0.030
	UTK-FairFace [7]	86.43 (0.009)	79.38 (0.007)	78.40 (0.010)	84.44 (0.011)	83.87 (0.011)	0.042	0.034
	Face4FairShifts	75.06 (0.002)	74.50 (0.005)	75.90 (0.015)	78.49 (0.004)	79.28 (0.014)	0.026	0.021
Age Race	UTKFace [88]	76.90 (0.018)	76.37 (0.005)	75.44 (0.010)	78.81 (0.007)	77.24 (0.003)	0.015	0.012
	FairFace [38]	87.42 (0.005)	84.23 (0.007)	82.71 (0.007)	88.90 (0.003)	88.89 (0.004)	0.034	0.028
	UTK-FairFace [7]	84.67 (0.008)	81.63 (0.009)	81.62 (0.004)	85.60 (0.004)	85.19 (0.004)	0.023	0.020
	Face4FairShifts	73.53 (0.004)	73.97 (0.003)	74.70 (0.004)	76.37 (0.005)	75.30 (0.004)	0.014	0.011

Label	Dataset	$\Delta \text{DP}$
Sen.		LFR	GSR	AD	CSAD	FNF	$ϵ$	$\pm$
Age Gender	Celeba [48]	0.062 (0.007)	0.069 (0.002)	0.067 (0.003)	0.049 (0.002)	0.047 (0.002)	0.012	0.010
	UTKFace [88]	0.021 (0.001)	0.020 (0.004)	0.021 (0.002)	0.020 (0.001)	0.019 (0.003)	0.001	0.001
	FairFace [38]	0.071 (0.003)	0.025 (0.006)	0.045 (0.007)	0.025 (0.004)	0.026 (0.005)	0.023	0.020
	UTK-FairFace [7]	0.045 (0.004)	0.019 (0.001)	0.021 (0.000)	0.018 (0.000)	0.018 (0.001)	0.011	0.011
	Face4FairShifts	0.092 (0.001)	0.083 (0.003)	0.083 (0.002)	0.079 (0.002)	0.071 (0.002)	0.009	0.008
Age Race	UTKFace [88]	0.096 (0.004)	0.121 (0.006)	0.100 (0.003)	0.083 (0.005)	0.097 (0.003)	0.016	0.014
	FairFace [38]	0.144 (0.004)	0.134 (0.005)	0.129 (0.007)	0.126 (0.004)	0.123 (0.002)	0.010	0.008
	UTK-FairFace [7]	0.122 (0.007)	0.094 (0.005)	0.098 (0.005)	0.092 (0.005)	0.095 (0.005)	0.013	0.013
	Face4FairShifts	0.154 (0.006)	0.150 (0.006)	0.155 (0.005)	0.145 (0.004)	0.143 (0.006)	0.006	0.005

Label	Dataset	$\Delta \text{EO}$
Sen.		LFR	GSR	AD	CSAD	FNF	$ϵ$	$\pm$
Age Gender	Celeba [48]	0.062 (0.006)	0.051 (0.015)	0.051 (0.015)	0.036 (0.002)	0.036 (0.003)	0.013	0.011
	UTKFace [88]	0.063 (0.002)	0.070 (0.009)	0.070 (0.010)	0.053 (0.002)	0.052 (0.003)	0.011	0.009
	FairFace [38]	0.083 (0.002)	0.064 (0.003)	0.065 (0.004)	0.059 (0.002)	0.060 (0.001)	0.010	0.010
	UTK-FairFace [7]	0.047 (0.002)	0.040 (0.005)	0.042 (0.004)	0.033 (0.002)	0.030 (0.002)	0.009	0.007
	Face4FairShifts	0.086 (0.005)	0.083 (0.004)	0.083 (0.004)	0.074 (0.002)	0.070 (0.002)	0.008	0.007
Age Race	UTKFace [88]	0.098 (0.010)	0.088 (0.001)	0.089 (0.001)	0.084 (0.001)	0.087 (0.002)	0.006	0.005
	FairFace [38]	0.129 (0.003)	0.120 (0.002)	0.123 (0.007)	0.113 (0.001)	0.119 (0.001)	0.007	0.006
	UTK-FairFace [7]	0.102 (0.008)	0.086 (0.003)	0.088 (0.001)	0.086 (0.005)	0.087 (0.002)	0.007	0.007
	Face4FairShifts	0.142 (0.003)	0.136 (0.003)	0.142 (0.004)	0.136 (0.003)	0.132 (0.003)	0.006	0.005

Equations

$$\text{DI}=k,\text{if DI}\le 1;\text{DI}=1/k,\text{otherwise},\text{where }k=\frac{\mathbb{P}(Y=1|Z=-1)}{\mathbb{P}(Y=1|Z=1)}$$

$$\text{Accuracy}=\frac{1}{N}\sum _{i=1}^{N}1[y_i^{pred}=y_i^{true}]$$

$$\text{Jaccard Accuracy}=\frac{1}{N}\sum _{i=1}^N\frac{|y_i^{pred}\cap y_i^{true}|}{|y_i^{pred}\cup y_i^{true}|}$$

$$\displaystyle D_{JS}=\frac{1}{2}\sum\nolimits_{i=1}^{M}\sum\nolimits_{j=1}^{M}\Big{(}\frac{1}{2}KL(D_{i}||\frac{D_{i}+D_{j}}{2})+\frac{1}{2}KL(D_{j}||\frac{D_{i}+D_{j}}{2})\Big{)},\forall i\neq j$$

$$\Delta \text{DP}=|\mathbb{P}(\hat{Y}=1|Z=1)-\mathbb{P}(\hat{Y}=1|Z=-1)|$$

$$\Delta \text{EO}={\sum }_{y\in \{0,1\}}|\mathbb{P}(\hat{Y}=1|Z=1,Y=y)-\mathbb{P}(\hat{Y}=1|Z=-1,Y=y)|$$

$$ϵ=\frac{1}{2}{\sum }_{i=1}^N{\sum }_{j=1}^N|{\text{baseline}}_i-{\text{baseline}}_j|,\forall i\ne j$$

Full text

1 Introduction

Distribution shifts between source and target domains present a fundamental challenge in machine learning, affecting model generalization in real-world applications. For example, a facial recognition system trained on high-resolution studio portraits may perform poorly when applied to low-quality surveillance footage, leading to accuracy degradation and biased predictions. Such shifts in visual characteristics highlight the need for robust adaptation techniques to ensure model reliability. Broadly, as shown in Table 1, distribution shifts can be categorized into five key types: (1) covariate shifts shimodaira2000improving , (2) semantic shifts wang2003mining , (3) demographic shifts giguere2022fairness , where the marginal distribution of input features, class labels, and sensitive attributes (race and gender) changes across domains, respectively; (4) concept shifts conceptshift , where the relationship between input features and labels changes; and (5) correlation shifts roh2023improving ; fedora , where spurious correlations between sensitive attributes and class labels vary across domains. These shifts, whether occurring independently or in combination, can significantly impact both predictive performance and fairness, making them critical considerations in real-world deployment.

Recently, fairness-aware out-of-distribution (OOD) generalization (FairOG) fedora ; fatdm ; flair ; utk-fairface ; fvae ; eiil , has emerged as an important research area, addressing the joint effects of covariate and correlation shifts. The FairOG setting assumes access to training data from multiple source domains and aims to learn a fair classifier that generalizes to an unseen and inaccessible target domain while maintaining both predictive accuracy and fairness. However, studying FairOG in practice is challenging due to the

lack of suitable benchmark datasets. Existing datasets utk-fairface ; fairface are often repurposed by artificially partitioning a single dataset into multiple domains fedora or by defining sensitive attributes post hoc eiil ; fatdm . These adaptations frequently fail to capture meaningful domain-specific covariate shifts or strong correlations between sensitive attributes and class labels, limiting their effectiveness in evaluating FairOG methods. A detailed discussion of related benchmark datasets is provided in Appendix A. To enable rigorous research in FairOG, there is a pressing need for a dataset that naturally exhibits covariate shifts across multiple domains while simultaneously demonstrating correlation shifts between sensitive attributes and class labels. Such a dataset would provide a more realistic and robust testbed for developing and evaluating fairness-aware generalization techniques under real-world distribution shifts.

In this paper, we introduce, Face4FairShifts, a large-scale facial image benchmark designed to study fairness-aware domain generalization and robustness under distribution shifts. Face4FairShifts consists of 100,000 facial images spanning four visually distinct domains, Photo, Art, Cartoon, and Sketch, each exhibiting meaningful covariate shifts. The dataset is constructed using a combination of existing datasets and web-crawled images, ensuring diverse and representative domain characteristics. Each image in Face4FairShifts is annotated with 42 annotations across 15 attributes, capturing various demographic and facial characteristics. To ensure high-quality annotations, we employed 66 human annotators and a dedicated 5-person quality control team to refine and validate the labels. We conduct extensive experiments to evaluate Face4FairShifts across four key machine learning research areas: fairness learning, OOD generalization, OOD detection, and FairOG. In each area, we compare Face4FairShifts with existing datasets that have been adapted for the corresponding problem setting and evaluate it against state-of-the-art methods to ensure a comprehensive benchmark. Our results show that Face4FairShifts presents significant challenges to current baseline models, highlighting the difficulties of achieving fairness and robustness under realistic domain shifts. By providing a carefully curated benchmark with meaningful distribution shifts, Face4FairShifts serves as a valuable resource for advancing research in fair and robust machine learning. The main contribution of this paper is summarized:

• •

We introduce Face4FairShifts, a novel large-scale facial dataset specifically designed for studying fairness and robust machine learning. The dataset comprises 100,000 images spanning four distinct visual domains and includes 42 annotations across 15 attributes, capturing a diverse range of facial features.

• •

Images in Face4FairShifts are sourced from existing datasets and crawled from search engines. We recruit 66 paid annotators to perform over 12.5 million annotation instances and a 5-person team to refine and validate the labels, making the annotation process highly labor-intensive, and requiring substantial effort and resources.

• •

Extensive empirical studies on various state-of-the-art baseline methods across four research areas, including fairness learning, OOD generalization, OOD detection, and fairness-aware OOD generalization, demonstrate that Face4FairShifts presents greater challenges compared to existing datasets while exhibiting more stable performance across baseline methods. These findings establish Face4FairShifts as a valuable resource for developing and evaluating fair and robust machine learning techniques.

2 The Face4FairShifts Dataset

2.1 Data Collection

Our Face4FairShifts dataset collection process consists of two steps. First, we gather publicly available datasets and apply filtering to ensure data quality and relevance. Second, if a domain lacks sufficient data after filtering, we supplement it by crawling images from search engines. Specifically, for the four domains (Photo, Art, Cartoon, and Sketch) in Face4FairShifts, all images in the Photo and Cartoon domains come from existing datasets, while the majority in the Art and Sketch domains are obtained through web crawling due to the limited availability of public datasets. This approach ensures a diverse and comprehensive dataset while maintaining consistency across domains.

Collection from Existing Facial Datasets. We systematically collect existing datasets and carefully select the most suitable ones to construct each domain of Face4FairShifts. These datasets include not only facial image datasets but also full-body and half-body image datasets, which require cropping to align with our requirements. We found that no existing facial dataset spans all four designed domains, highlighting the significance and necessity of the Face4FairShifts dataset.

• •

Photo Domain. CelebA CelebA is a large-scale celebrity face dataset with rich annotations, featuring a diverse range of images across different races, age groups, and other characteristics. Its diversity makes it well-suited for representing face images in the Photo domain. Based on this, we select 30,000 images from CelebA. Additionally, the detailed annotations in CelebA serve as valuable references for Face4FairShifts.

• •

Art Domain. MetFaces karras2020training is a face dataset extracted from artwork, comprising 1,336 high-quality images obtained via the Metropolitan Museum of Art Collection Application Programming Interface API . To maintain a clear distinction between the Art and Sketch domains, we exclude black-and-white images from this dataset. The WikiART artgan2018 dataset contains 6,095 portrait face images scraped using the WikiART crawler artgan2018 , covering a diverse range of artistic styles, with black-and-white images similarly excluded. The Human-Art ju2023human dataset, proposed by the Chinese University of Hong Kong, is a multifunctional dataset featuring images across various categories, with 2,000 images per category. For the Art domain in Face4FairShifts, we include images from the watercolor and oil painting categories.

• •

Cartoon Domain. IIIT-CFW mishra2016iiit is a cartoon face dataset containing 8,928 images of 100 world-renowned individuals from various professions. Due to variations in image size, with some being small and low-resolution, we only utilize a carefully selected subset of high-quality images. iCartoonFace zheng2020cartoon is a large-scale face recognition dataset comprising 389,678 images of 5,013 cartoon characters collected from public websites and online videos. The majority of the images in the Cartoon domain of Face4FairShifts come from iCartoonFace.

• •

Sketch Domain. The CUFSF zhang2011coupled dataset includes face images with lighting variation and one corresponding sketch for each of its 1,194 individuals. We include these 1,194 sketches in the Sketch domain of Face4FairShifts. FS2K fan2022facial is a high-quality facial sketch dataset consisting of 2,104 sketch images created by professional artists in three distinct styles. Overall, the available data for the Sketch domain remains limited.

Image Crawling via Search Engines. After collecting existing datasets, the total number of images remained significantly below our target (100K). To address this, we developed a Python-based web crawler to gather additional images from multiple search engines, including Google and Bing. The images were primarily sourced from various platforms such as Xiaohongshu, YouTube, Douyin, Saatchi Art, Etsy, eBay, Blibli, and Sina.

2.2 Data Processing

In the collected data, particularly those obtained through search engines, it is inevitable to encounter images that either do not contain any individuals or feature multiple people. For person detection, we utilize the YOLOv5 object detection tool glenn_jocher_2020_3983579 , which allows us to systematically process and filter the collected images. It removes images where no person is detected and segments each detected individual into a separate image, ensuring that each output image contains only one person. Specifically, for each crawled image, we focus exclusively on detecting the "person" class, setting a confidence threshold of 0.7. If the confidence score for detecting a person in an image falls below this threshold, the object is disregarded, while those meeting the criterion are retained. However, this process introduces a new challenge: some of the segmented individuals are too small, making them unsuitable for our dataset. To mitigate this issue, we manually determine a minimum acceptable image size as a reference. We then iterate through all cropped images and discard those that fall below this size threshold, ensuring sufficient clarity and quality. Additionally, since the cropped images of detected individuals may include half-body or full-body portraits, and automated face cropping may not be reliable for achieving consistent image framing, manual cropping using Photoshop is required. Although labor-intensive, this final step was necessary to ensure high-quality, standardized image framing across the dataset, maintaining consistency for subsequent analysis. Figure 2 (Left) illustrates the dataset statistics following the data processing stage.

2.3 Data Annotation

Each processed face image in Face4FairShifts is annotated with 42 binary labels across 15 attributes, covering a diverse range of features. Our annotation scope is extensive, encompassing sensitive attributes related to fairness, such as gender, race, and age, as well as detailed facial attributes including beard, glasses, eyes, mouth, smile, nose, chin, hair, braids, accessories, and others. To enhance the dataset’s versatility and enable fine-grained analysis, each facial attribute is further divided into multiple specific annotations, such as male, female, Asian, black, and so forth. All annotations in Face4FairShifts are manually labeled to ensure high quality. An example in Figure 7 of the Appendix illustrates the observed and unobserved labels for a given image in the Art domain. We recruited 66 paid annotators and provided them with comprehensive training sessions to ensure a thorough understanding of the annotation guidelines. To maintain consistency and accuracy, each image is labeled by at least three annotators, and the final annotation was determined using a majority voting mechanism, where the label agreed upon by the majority was assigned as the final decision. To further enhance data quality, a five-person quality control team conducted random sample reviews to verify annotation accuracy. The total number of annotation instances in the Face4FairShifts dataset exceeds 12,600,000, making this a highly labor-intensive process that requires substantial effort and resources. Figure 1 showcases example facial images, and Figure 3 summarizes the distribution of 42 annotations across the four domains.

Furthermore, we conduct a comparative analysis between human annotations and those produced by large language models (LLMs). We randomly selected 500 images from each of the four domains, resulting in a total of 2000 images. These images were then input into four representative LLMs: GPT-4o, Claude 3.7 Sonnet, Gemini 2.5, and Kimi Latest. A carefully designed prompt was constructed to guide the models in performing visual attribute annotation. Figure 2 (Right) presents the performance comparison of the four LLMs in terms of accuracy and F1-score across all 15 facial attribute categories, demonstrating the consistency and reliability of human annotations as a benchmark. More details of data annotations refer to Appendix 8.

2.4 Fairness Statistics

To evaluate the fairness characteristics of Face4FairShifts, we perform a statistical analysis across all four domains. This analysis aims to quantify fairness-related biases and disparities both within each domain and the dataset as a whole, providing insights into how fairness varies across different combinations of sensitive attributes and labels. Specifically, we examine the absolute correlation between all 14 attributes using heatmaps, which visually represent how these attributes interact within each combination. Furthermore, we compute disparate impacts (DI) to assess fairness disparities. By measuring the degree to which positive class labels ($Y=1$) are influenced by sensitive attributes ($Z$), we quantify whether certain sensitive groups are disproportionately advantaged or disadvantaged. For this analysis, we discretize these attributes into binary groups, defining race as Black and non-Black, age as elderly and non-elderly, beard as has-beard and no-bread, glasses as has-glasses and no-glasses, hairs as bald and no-bald, braids as has-braids and no-braids, and accessory as has-hat and no-hat. The disparate impact is then calculated as:

[TABLE]

Following the "80%-rule" biddle2017adverse , DI values greater than or equal to 0.8 indicate that the data meets fairness criteria. Figure 4 highlights substantial correlation shifts between sensitive attributes and class labels across domains, demonstrating the challenges of fair model adaptation when data distributions shift. These findings suggest that models trained on one domain may exhibit unfair biases when generalized to another domain, emphasizing the need for fairness-aware domain adaptation techniques. As a result, Face4FairShifts serves as a valuable benchmark for fairness-aware domain generalization, providing a robust testbed for studying bias mitigation techniques under both covariate and correlation shifts. By exposing fairness disparities in real-world facial datasets across multiple visual styles, Face4FairShifts facilitates the development of more equitable, unbiased, and robust machine learning models.

3 Experiments

To assess the effectiveness and robustness of Face4FairShifts, we conduct extensive experiments across four key machine learning research areas: fairness learning, OOD generalization, OOD detection, and FairOG. In each area, we compare Face4FairShifts with existing datasets that have been adapted for the corresponding problem setting and evaluate it against state-of-the-art methods to ensure a comprehensive benchmark. More details of experiment settings and results refer to Appendix C. These experiments aim to address the following questions:

How does Face4FairShifts capture and reflect the impact of covariate shifts across different domains compared to existing facial datasets? (Answers: Section 3.1 and Figure 6) 2. 2.

Compared to existing datasets across four different machine learning problems (fairness learning, OOD generalization, OOD detection, and FairOG), does Face4FairShifts pose greater challenges for state-of-the-art baseline methods, thereby providing more room for improvement and future research? (Answers: Sections 3.2, 3.3, 3.4, 3.5 and Tables 3, 4, 5, 6, 7, 8) 3. 3.

Compared to existing datasets across four different machine learning problems, does Face4FairShifts lead to more consistent performance across state-of-the-art baseline methods while providing a more stable benchmark for evaluation? (Answers: The $\epsilon$ and $\pm$ indicators in Tables 3, 4, 5, 6, 7, 8)

Datasets. (1) CelebA CelebA is a large-scale facial attributes dataset comprising 200,599 celebrity images, each annotated with 40 binary attributes. While it has been widely utilized for fairness learning, it lacks race annotations. Additionally, it has rarely been employed for investigating distribution shifts. (2) UTKFace utk is a facial dataset with a wide age span, consisting of over 20,000 face images annotated with age, gender, and race, including White, Black, Asian, Indian, and Others. Similar to CelebA, this dataset has rarely been utilized for studying distribution shifts. (3) FairFace fairface is a facial dataset designed to provide a balanced representation across racial groups. It comprises 108,501 images spanning seven racial categories: Black, East Asian, Indian, Latino, Middle Eastern, Southeast Asian, and White. Following the approach in fedora , the seven racial groups are treated as distinct domains. (4) UTK-FairFace utk-fairface is a semi-synthetic dataset created by combining UTKFace and FairFace. In utk-fairface , the authors argued that a distribution shift exists between the two datasets due to differences in image sources, leading to the consideration of each dataset as a separate domain. However, this dataset primarily relies on source discrepancies rather than naturally occurring domain shifts. An overview of all datasets used for experiments is given in Table 2.

Due to space constraints, we defer the detailed introduction of baseline methods and implementation specifics for each research area (Sections 3.2, 3.3, 3.4, and 3.5) to the Appendix. All results presented in Tables 3, 4, 5, 6, 7, and 8 are averaged over three repeated runs.

3.1 Feature Space Analysis

We perform a preliminary analysis to contrast the covariate shifts within our Face4FairShifts to that of prior used facial datasets for FairOG, such as FairFace in fedora and UTK-Fairface in utk-fairface . Figure 6 presents the t-SNE visualization of data samples across different domains, utilizing ResNet50 he2016deep and ViT-Base dosovitskiy2020image as feature extraction backbones pre-trained on ImageNet deng2009imagenet . Additionally, we quantify the covariate shift across domains for all datasets using the Jensen-Shannon (JS) divergence endres2003new , which is computed based on the Kullback-Leibler (KL) divergence. Our results clearly demonstrate that the covariate shift in Face4FairShifts is significantly higher compared to other datasets, highlighting the dataset’s effectiveness in modeling real-world distributional variations.

3.2 Fairness Learning

Setting. To evaluate model fairness, we treat each dataset as a whole, disregarding domain distinctions. Specifically, we consider two label-sensitive settings: age-gender and age-race. In these settings, age is discretized into elderly and non-elderly, gender is categorized as male and female, and race is grouped into Black and non-Black. This approach allows us to assess fairness disparities across sensitive attributes without the influence of domain shifts.

Evaluation Metrics. We employ two widely used fairness evaluation metrics: the absolute difference of demographic difference ($\Delta$DP) dwork2012fairness and that of equalized odds ($\Delta$EO) hardt2016equality . A value of zero for these metrics indicates fair predictions, meaning the model’s decisions are independent of the sensitive attribute $Z$. Additionally, to analyze the stability of these baselines on a given dataset, we introduce two stability indicators, $\epsilon$ and $\pm$. $\epsilon$ quantifies the average absolute difference between any two baseline methods. The $\pm$ indicator represents the standard deviation of the performance across all baseline methods. Smaller values of both $\epsilon$ and $\pm$ indicate more stable and consistent performance across different baselines.

Baseline Methods. We evaluate the performance of all datasets using five widely adopted baseline methods, including LFR lfr , GSR gsr , AD ad , CSAD csad , and FNF fnf .

Results. Table 3 illustrates that Face4FairShifts presents greater challenges compared to all other datasets, as evidenced by the lowest accuracy and the highest values of $\Delta$DP and $\Delta$EO across the majority of baseline methods. Table 10 in the Appendix provides a more detailed version of Table 3, including standard deviations for each baseline method. Furthermore, Face4FairShifts exhibits more stable performance across baseline methods, as indicated by the consistently lower values of $\epsilon$ and $\pm$ across all metrics. Note that since CelebA does not include race attributes, it is excluded from the age-race setting in our experiments.

3.3 OOD Generalization

Setting. In OOD generalization, since target domains are unknown and inaccessible during training, followed mbdg ; gulrajani2020search , we use leave-one-domain-out validation criteria. Specifically, we evaluate selected datasets, FairFace, UTK-Fairface, and Face4FairShifts, by holding out one domain during training and assessing model performance on the held-out one (target). The final performance is reported as the average across all held-out domains, ensuring a robust evaluation of generalization under distribution shifts.

Baseline Methods. The performance of all datasets is evaluated using six widely recognized methods ERM erm , IRM irm , GDRO gdro , Mixup mixup , MMD mmd , MBDG mbdg . These methods are implemented in the DomainBed repository gulrajani2020search , a widely used benchmarking framework for domain generalization, which has gained significant traction, accumulating over 1.5k stars on GitHub.

Results. Similar to the fairness learning results in Section 3.2, Table 4 highlights that Face4FairShifts presents greater challenges compared to other datasets, as indicated by the lowest accuracy and F-1 scores across various baseline methods. Additionally, it demonstrates more stable performance, evidenced by the consistently smaller values of $\epsilon$ and $\pm$.

3.4 OOD Detection

Setting. In OOD detection, we consider three distinct scenarios, as illustrated in Figure 9 in the Appendix. In inter-domain sensory OOD detection, images exhibiting covariate shifts are considered OOD. In intra-domain and inter-domain semantic OOD detection, images with semantic shifts are identified as OOD. Specifically, in inter-domain semantic OOD detection, only images with covariate shifts are regarded as in-distribution (InD), while those with semantic shifts are classified as OOD. Detailed experiment settings are provided in Section C.4 of the Appendix.

Evaluation Metrics. We evaluate OOD detection performance across all scenarios using AUROC davis2006relationship and AUPR manning1999foundations . AUROC measures the trade-off between the true positive rate and the false positive rate, providing an overall assessment of a model’s ability to distinguish between InD and OOD samples. AUPR evaluates performance based on precision and recall, which are particularly useful in imbalanced settings where OOD samples are rare. Notice that the accuracy (InD/OOD for sensory, InD for intra- and inter-domain semantic) is provided for reference only.

Baseline Methods. For inter-domain sensory OOD detection and intra-domain semantic OOD detection, we use One-class SVM (ocSVM) oc-svm , DDU ddu , MSP msp , Energy energy and Entropy entropy as baseline methods. For inter-domain semantic OOD detection, we use EDst edst , SCONE scone , DAML daml , MEDIC medic , MADOD maood as baseline methods. These methods provide a diverse range of approaches for evaluating OOD detection performance under both covariate and semantic distribution shifts.

Results Table 5 shows that Face4FairShifts achieves the best performance in inter-domain sensory OOD detection, outperforming FairFace and UTK-FairFace in terms of AUROC, AUPR, and InD/OOD accuracy. In this setting, images exhibiting covariate shifts are treated as OOD. The results in this table align with the trends observed in Figure 6, further supporting the effectiveness of Face4FairShifts. Additionally, Tables 6 and 7 present the results for the settings of intra- and inter-domain OOD detection. Again, Face4FairShifts poses challenges across all settings compared to FairFace and UTK-FairFace, demonstrating consistently lower performance across different baseline methods while maintaining stability across them.

3.5 Fairness-aware OOD Generalization

Setting. Similar to the settings used in fairness learning and OOD generalization, we evaluate FairFace, UTK-FairFace, and our Face4FairShifts across multiple domains in the FairOG setting. We consider two label-sensitive settings to examine fairness disparities while accounting for the impact of domain shifts. This approach provides a comprehensive evaluation of how fairness-aware models generalize under distributional variations. To ensure a robust assessment, the final performance is reported as the average across all held-out domains.

Baseline Methods. We evaluate all datasets using six recently published methods, FCR utk-fairface , EIIL eiil , FVAE fvae , FATDM fatdm , FLAIR flair , and FEDORA fedora .

Results. Since the race attribute is used to define domains in fedora , we exclude FairFace from the age-race setting in this experiment. Table 8 further demonstrates that Face4FairShifts presents greater challenges compared to other datasets under both settings while maintaining stable performance across various state-of-the-art methods.

4 Conclusion

In this paper, we introduce Face4FairShifts, a large-scale facial image benchmark designed to study fairness and robustness under domain shifts. It spans four visual domains, comprising 100,000 images with 42 annotations across 15 attributes. Extensive experiments demonstrate that Face4FairShifts serves as a valuable resource for developing and evaluating fair and robust machine learning techniques.

Appendix A Related Work

Table 9 presents a list of related real-world image datasets focused on facial attributes and distribution shifts.

Datasets for Fairness Learning. Fairness-aware machine learning has been extensively studied using both tabular and image datasets that capture demographic disparities. Several widely used tabular datasets, such as Adult Income [1], COMPAS [3], German Credit [4], and Bank Marketing [2], are frequently used to evaluate fairness learning algorithms due to their inclusion of sensitive attributes like race and gender. In facial image datasets, studies often rely on CelebA [48], which provides extensive facial attribute annotations but is imbalanced in demographic representation. More balanced alternatives, including UTKFace [88] and FairFace [38], offer a diverse distribution of race, age, and gender, making them more suitable for fairness-aware classification. Another dataset, WaterBird [65], has been used to study spurious correlations between background features and class labels, serving as an example of how dataset biases can affect classification models. However, while these datasets facilitate fairness research, they lack inherent covariate shifts and thus do not adequately support studies on FairOG.

Datasets for Distribution Shifts. To study distribution shifts, researchers have leveraged datasets designed to capture variations in visual domains. Multi-domain benchmarks such as VLCS [74], which combines images from existing datasets [23, 75, 25, 14], introduce natural distribution shifts for object recognition. Similarly, PACS [44], containing images categorized into Photo, Art, Cartoon, and Sketch domains, is widely used to evaluate robustness to style variations. Additionally, datasets like Office [63], composed of images collected from sources such as Amazon, DSLR, and Webcam, enable research on domain adaptation in classification tasks. For large-scale multi-domain learning, DomainNet [56] provides images from six diverse domains, making it a comprehensive resource for studying robustness across visual shifts. Beyond visual style and object recognition, datasets included in WILDS [41], such as Camelyon17 [41] for histopathology analysis and FMoW [41] for satellite imagery, have been used to evaluate distribution shifts in medical and remote sensing applications. Although these datasets effectively model covariate and semantic shifts, they do not explicitly include fairness-sensitive attributes, limiting their application in FairOG.

Datasets for Fairness-aware Distribution Shifts. FairOG remains an emerging research area, yet existing datasets rarely address both fairness and robustness under distribution shifts. Some studies [7, 89, 45] have attempted to modify fairness-focused datasets such as UTKFace and FairFace by partitioning them into synthetic domains or manipulating sample distributions to simulate artificial shifts. However, these modifications, such as using the combined UTK-FairFace [7] dataset or partitioning FairFace into seven domains based on race [89], often fail to capture meaningful domain-specific covariate shifts or establish strong correlations between sensitive attributes and class labels. The absence of a dedicated dataset that simultaneously encapsulates fairness-sensitive attributes and realistic distribution shifts underscores the need for a benchmark tailored to FairOG. Face4FairShifts addresses these limitations by naturally incorporating both fairness-sensitive attributes and diverse domain shifts, providing a well-structured benchmark for studying FairOG.

Appendix B More Details of Data Annotation

• •

Figure 7 illustrates an example of data annotation for a sample image.

• •

The prompt template used to generate 15 attribute annotations with LLMs for input images sampled from different domains is shown in Figure 8.

• •

During data annotation, we perform a comparative analysis between human annotations and those generated by large language models (LLMs), including GPT-4o, Claude 3.7 Sonnet, Gemini 2.5, and Kimi Latest. The comparison is conducted across 15 facial attributes using accuracy and F1-score, with human annotations serving as the ground truth. Specifically, for single-label attributes such as gender, race, age, and appearance, accuracy is computed as:

[TABLE]

For the remaining multi-label attributes, accuracy is evaluated using the Jaccard accuracy [53] (also known as the Jaccard Index or Intersection over Union):

[TABLE]

Here, the numerator represents the number of correctly predicted labels, while the denominator denotes the total number of unique labels from both the prediction and the ground truth.

Appendix C Experiment Details

All experiments are conducted on NVIDIA A100 and NVIDIA GeForce RTX 3090 GPUs, and the results are reported as the average performance over 3 runs.

C.1 Feature Space Analysis

Evaluation Metrics. We quantify the covariate shift across domains for all datasets using the Jensen-Shannon (JS) divergence [21], which is computed based on the Kullback-Leibler (KL) divergence as follows.

[TABLE]

where $D_{i}$ and $D_{j}$ are data features from any two different domains and $M$ denotes the number of domains.

C.2 Fairness Learning

Evaluation Metrics. We employ two widely used fairness evaluation metrics: the absolute difference of demographic difference ($\Delta$DP) [19] and that of equalized odds ($\Delta$EO) [30].

[TABLE]

where $Y$ represents the ground-truth class label and $\hat{Y}$ denotes the predicted label. A value of zero for these metrics indicates fair predictions, meaning the model’s decisions are independent of the sensitive attribute $Z$. Additionally, to analyze the stability of these baselines on a given dataset, we introduce two stability indicators, $\epsilon$ and $\pm$. $\epsilon$ quantifies the average absolute difference between any two baseline methods.

[TABLE]

where $N$ denotes the number of baseline methods. The $\pm$ indicator represents the standard deviation of the performance across all baseline methods. Smaller values of both $\epsilon$ and $\pm$ indicate more stable and consistent performance across different baselines.

We conducted fairness learning experiments on the CelebA, UTKFace, FairFace, UTK-FairFace, and Face4FairShifts datasets, selecting age as the binary label. In Face4FairShifts, age is discretized into "elderly" and "non-elderly". For datasets where age is a continuous variable (e.g., UTKFace and FairFace), we binarize the labels by setting the threshold at 50 years old. We then conducted experiments using gender and race as binary sensitive attributes, where race was standardized into "black" and "non-black". Note that since CelebA does not contain race information, we excluded it from experiments where race was the sensitive attribute. Since this experiment does not involve domain partitioning, each dataset was randomly split into training and test sets in an 8:2 ratio. Each experiment was repeated three times, and the results were averaged. The fairness learning results, along with standard deviations, are reported in Table 10.

We evaluate the performance of all datasets in fairness learning using five widely adopted baseline methods: LFR [84], GSR [6], AD [85], CSAD [92], and FNF [10].

• •

LFR [84] aligns distributions across protected groups to create fair representations while preserving data utility and demographic parity.

• •

GSR [6] converts fairness constraints into classification error bounds via sample reweighting and margin adjustments.

• •

AD [85] removes bias through adversarial networks that suppress sensitive attribute correlations in features.

• •

CSAD [92] minimizes cross-sample mutual information to disentangle bias-invariant patterns from latent representations.

• •

FNF [10] generates fair data via invertible flow models that transform biased latent distributions.

For methods like LFR and GSR that do not directly process image data, we extract features from images using a ResNet50 pre-trained on ImageNet and use these extracted features as model inputs. The implementations of LFR, GSR, and AD are based on the AIF360 repository111https://github.com/Trusted-AI/AIF360, while CASD and FNF are implemented using their publicly available code.

C.3 OOD Generalization

We conducted domain generalization experiments on the FairFace, UTK-FairFace, and Face4FairShifts datasets, following the domain split settings of  [7, 89] for FairFace and UTK-FairFace. In this experiment, we used age and gender as binary classification targets. We adopted a leave-one-domain-out validation criterion, where for each domain in the dataset, we performed the following experiment three times: training on data from all other domains and testing on the held-out domain. The final result for each domain was obtained by averaging the three runs, and the overall experiment result was derived by averaging across all domains.

The performance of all datasets is evaluated using six widely recognized methods ERM [76], IRM [8], GDRO [64], Mixup [83], MMD [46], MBDG [59].

• •

ERM [76] minimizes empirical training loss to achieve generalization.

• •

IRM [8] learns invariant predictors across environments by enforcing feature-to-label invariance in causal mechanisms.

• •

GDRO [64] optimizes worst-case group performance via distributionally robust objectives and regularization for group shifts.

• •

Mixup [83] augments data with linear interpolations of samples to encourage smoother decision boundaries and reduce spurious correlations.

• •

MMD [46] aligns feature distributions across domains using maximum mean discrepancy minimization in kernel space.

• •

MBDG [59] synthesizes diverse training domains via generative models to improve adaptation to unseen target domains.

The MBDG method was implemented using its publicly available code, while other methods were implemented using the DomainBed repository [28].

C.4 OOD Detection

We conducted experiments on inter-domain sensory, intra-domain semantic, and inter-domain semantic OOD detection using the FairFace, UTK-FairFace, and Face4FairShifts datasets, following the same domain partitioning criteria as in OOD generalization. The experimental settings are illustrated in Figure 9. Specifically, we categorized age into three classes (teenager, middle, and elderly) and treated images of elderly individuals as semantic OOD samples. For inter-domain sensory and inter-domain semantic OOD detection tasks, we adopted the leave-one-domain-out validation criteria, where images from the held-out domain were considered covariate OOD samples. In contrast, for intra-domain semantic OOD detection, we treated the entire dataset as a single domain. For the OOD detection task, the training set consisted of 80% of the data within the training data range shown in Figure 9, while the remaining 20%, along with all other data, formed the test set.

For inter-domain sensory OOD detection and intra-domain semantic OOD detection, we use One-class SVM (ocSVM) [62], DDU [52], MSP [32], Energy [47] and Entropy [77] as baseline methods. For inter-domain semantic OOD detection, we use EDst [54], SCONE [9], DAML [69], MEDIC [81], MADOD [79] as baseline methods.

• •

ocSVM [62] detects OOD samples by learning a hypersphere boundary around in-distribution data in a kernel-induced feature space.

• •

DDU [52] estimates uncertainty via deterministic deep kernel Gaussian processes to distinguish in- and out-of-distribution samples

• •

MSP [32] uses maximum softmax probability as a confidence score, flagging low-confidence predictions as OOD.

• •

Energy [47] leverages energy-based scores from logits to detect OOD inputs without requiring distributional assumptions.

• •

Entropy [77] identifies OOD samples by measuring prediction entropy, assuming higher uncertainty for anomalous inputs.

• •

EDst [54] employs domain-invariant feature alignment and simple statistical thresholds for open-domain generalization.

• •

SCONE [9] jointly optimizes OOD detection and generalization by leveraging unlabeled "wild" data for dual-purpose training.

• •

DAML [69] Enhances domain generalization via meta-learning with augmented domains to improve OOD detection robustness.

• •

MEDIC [81] learns dualistic meta-features to create generalizable decision boundaries for open-set domain shifts.

• •

MADOD [79] meta-trains models with invariance constraints to generalize OOD detection to unseen domains via g-invariance.

All ten OOD detection methods are implemented based on their original source code.

C.5 Fairness-aware OOD Generalization

We conducted fairness-aware OOD generalization experiments on the FairFace, UTK-FairFace, and Face4FairShifts datasets. Similar to the fairness experiments, we selected age as the binary label and chose gender and race as the binary sensitive attributes. The domain partitioning followed the same criteria as in the OOD generalization experiments, and we also adopted the leave-one-domain-out validation approach.

We evaluate all datasets using six recently published methods, FCR [7], EIIL [15], FVAE [55], FATDM [57], FLAIR [45], and FEDORA [89].

• •

FCR [7] enforces fairness consistency across domains via distributionally robust regularization to maintain equity under covariate shifts.

• •

EIIL [15] infers latent environments to learn invariant predictors that generalize fairly across unseen distributional shifts.

• •

FVAE [55] disentangles sensitive attributes via contrastive distribution alignment in variational autoencoders for bias-free OOD generalization.

• •

FLAIR [45] learns fairness-aware domain-invariant representations by disentangling content/style factors and suppressing sensitive attributes while preserving utility, addressing both covariate and correlation shifts simultaneously.

• •

FEDORA [89] addresses covariate shifts and correlation shifts simultaneously by learning fair invariant predictors through adversarial domain-invariant representation learning.

All five methods are implemented based on their original source code.

Appendix D Limitations and Discussion

While Face4FairShifts offers a significant advancement in benchmarking fairness and robustness under visual domain shifts, several limitations remain. First, despite efforts to ensure diversity, the dataset’s reliance on publicly available and web-crawled sources may introduce sampling biases or cultural imbalances that affect generalizability. Second, manual annotation, while thorough, may still suffer from subjectivity, particularly for attributes such as attractiveness, which are inherently ambiguous and context-dependent. Third, the current benchmark focuses primarily on facial imagery and may not generalize to broader fairness concerns in other modalities or domains. Future extensions could explore cross-modal fairness benchmarks, incorporate more fine-grained sensitive attributes, and develop automated tools to mitigate human annotation noise.

NeurIPS Paper Checklist

Claims 2. Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? 3. Answer: [Yes] 4. Justification: The abstract and introduction clearly present the motivation, contribution, and limitations of this study. 5. Guidelines:

• •

The answer NA means that the abstract and introduction do not include the claims made in the paper.

• •

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

• •

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

• •

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 6. 2.

Limitations 7. Question: Does the paper discuss the limitations of the work performed by the authors? 8. Answer: [Yes] 9. Justification: The paper discusses the limitations in Appendix D. 10. Guidelines:

• •

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

• •

The authors are encouraged to create a separate "Limitations" section in their paper.

• •

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

• •

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

• •

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

• •

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

• •

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

• •

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 11. 3.

Theory assumptions and proofs 12. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? 13. Answer: [N/A] 14. Justification: This paper is an experimental work without theoretical results. 15. Guidelines:

• •

The answer NA means that the paper does not include theoretical results.

• •

All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.

• •

All assumptions should be clearly stated or referenced in the statement of any theorems.

• •

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

• •

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

• •

Theorems and Lemmas that the proof relies upon should be properly referenced. 16. 4.

Experimental result reproducibility 17. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? 18. Answer: [Yes] 19. Justification: Detailed information required to reproduce the results is provided in the experimental sections and the Appendix. The code is attached, and we are committed to releasing it publicly upon acceptance. 20. Guidelines:

• •

The answer NA means that the paper does not include experiments.

• •

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

• •

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

• •

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

• •

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

1. (a)

If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. 2. (b)

If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. 3. (c)

If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset). 4. (d)

We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 21. 5.

Open access to data and code 22. Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? 23. Answer: [Yes] 24. Justification: Our paper provides open access to the data and code, along with sufficient details for reproducing the main experimental results. 25. Guidelines:

• •

The answer NA means that paper does not include experiments requiring code.

• •

Please see the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details.

• •

While we encourage the release of code and data, we understand that this might not be possible, so “No” is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

• •

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details.

• •

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

• •

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

• •

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

• •

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 26. 6.

Experimental setting/details 27. Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? 28. Answer: [Yes] 29. Justification: See Experiments and Appendix. 30. Guidelines:

• •

The answer NA means that the paper does not include experiments.

• •

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

• •

The full details can be provided either with the code, in appendix, or as supplemental material. 31. 7.

Experiment statistical significance 32. Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? 33. Answer: [Yes] 34. Justification: All experiments are conducted multiple times, and standard deviations are provided in tables. 35. Guidelines:

• •

The answer NA means that the paper does not include experiments.

• •

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

• •

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

• •

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

• •

The assumptions made should be given (e.g., Normally distributed errors).

• •

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

• •

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

• •

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

• •

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 36. 8.

Experiments compute resources 37. Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? 38. Answer: [Yes] 39. Justification: Experiments compute resources are provided in Appendix C. 40. Guidelines:

• •

The answer NA means that the paper does not include experiments.

• •

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

• •

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

• •

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper). 41. 9.

Code of ethics 42. Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 43. Answer: [Yes] 44. Justification: The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics. 45. Guidelines:

• •

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

• •

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

• •

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 46. 10.

Broader impacts 47. Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? 48. Answer: [N/A] 49. Justification: Only Technical reports 50. Guidelines:

• •

The answer NA means that there is no societal impact of the work performed.

• •

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

• •

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

• •

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

• •

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

• •

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 51. 11.

Safeguards 52. Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? 53. Answer: [N/A] 54. Justification: There is no such misuse risk. 55. Guidelines:

• •

The answer NA means that the paper poses no such risks.

• •

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

• •

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

• •

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 56. 12.

Licenses for existing assets 57. Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? 58. Answer: [Yes] 59. Justification: All datasets and models used are cited. 60. Guidelines:

• •

The answer NA means that the paper does not use existing assets.

• •

The authors should cite the original paper that produced the code package or dataset.

• •

The authors should state which version of the asset is used and, if possible, include a URL.

• •

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

• •

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

• •

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

• •

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

• •

If this information is not available online, the authors are encouraged to reach out to the asset’s creators. 61. 13.

New assets 62. Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? 63. Answer: [N/A] 64. Justification: The paper does not release new assets. 65. Guidelines:

• •

The answer NA means that the paper does not release new assets.

• •

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

• •

The paper should discuss whether and how consent was obtained from people whose asset is used.

• •

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 66. 14.

Crowdsourcing and research with human subjects 67. Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? 68. Answer: [N/A] 69. Justification: This paper does not involve crowdsourcing nor research with human subjects. 70. Guidelines:

• •

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

• •

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

• •

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 71. 15.

Institutional review board (IRB) approvals or equivalent for research with human subjects 72. Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? 73. Answer: [N/A] 74. Justification: This paper does not involve crowdsourcing nor research with human subjects. 75. Guidelines:

• •

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

• •

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

• •

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

• •

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 76. 16.

Declaration of LLM usage 77. Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required. 78. Answer: [N/A] 79. Justification: The core method development in this research does not involve LLMs. 80. Guidelines:

• •

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

• •

Please refer to our LLM policy (https://neurips.cc/Conferences/2025/LLM) for what should or should not be described.