Cinderella's shoe won't fit Soundarya: An audit of facial processing tools on Indian faces

TL;DR

This study evaluates the performance of four commercial facial processing tools on Indian faces, revealing significant errors in face detection, gender, and age classification, especially for women and certain age groups, highlighting limitations in current technology.

Contribution

The paper provides an empirical assessment of facial processing tools on Indian faces, exposing demographic biases and inaccuracies that are underexplored in existing research.

Findings

Gender classification error rate for Indian females up to 14.68%

Age prediction errors range from 14.3% to 42.2%

Tools show varying error rates and demographic biases

Abstract

The increasing adoption of facial processing systems in India is fraught with concerns of privacy, transparency, accountability, and missing procedural safeguards. At the same time, we also know very little about how these technologies perform on the diverse features, characteristics, and skin tones of India's 1.34 billion-plus population. In this paper, we test the face detection and facial analysis functions of four commercial facial processing tools on a dataset of Indian faces. The tools display varying error rates in the face detection and gender and age classification functions. The gender classification error rate for Indian female faces is consistently higher compared to that of males -- the highest female error rate being 14.68%. In some cases, this error rate is much higher than that shown by previous studies for females of other nationalities. Age classification errors are also high. Despite taking into account an acceptable error margin of plus or minus 10 years from a person's actual age, age prediction failures are in the range of 14.3% to 42.2%. These findings point to the limited accuracy of facial processing tools, particularly for certain demographic groups, and the need for more critical thinking before adopting such systems.