Has the Virtualization of the Face Changed Facial Perception? A Study of the Impact of Photo Editing and Augmented Reality on Facial Perception

TL;DR

This study investigates how different types of face modifications, including traditional filters and augmented reality filters, affect facial perception and recognition in online contexts, revealing perceptual differences based on filter type.

Contribution

The paper provides empirical data on how various facial filters influence perception and recognition, highlighting differences between traditional and augmented reality filters.

Findings

Traditional filters are perceived similarly to unmodified faces.

AR filters are perceived differently from unmodified faces.

Societal adaptation may influence perception of traditional filters.

Abstract

Augmented reality and other photo editing filters are popular methods used to modify faces online. Considering the important role of facial perception in communication, how do we perceive this increasing number of modified faces? In this paper we present the results of six surveys that measure familiarity with different styles of facial filters, perceived strangeness of faces edited with different filters, and ability to discern whether images are filtered. Our results demonstrate that faces modified with more traditional face filters are perceived similarly to unmodified faces, and faces filtered with augmented reality filters are perceived differently from unmodified faces. We discuss possible explanations for these results, including a societal adjustment to traditional photo editing techniques or the inherent differences in the different types of filters. We conclude with a discussion of how to build online spaces more responsibly based on our results.

Tables

Table 1. Table 1 . Are you familiar with this style of image? Statistical test results

Filter	Chi-Square Value	p-value
Flower Crown	855.64	$4.3\text{×}{10}^{-188}$
Fairy Lights	2353.62	0.0
Aviators	595.76	$1.40\text{×}{10}^{-131}$
Manly Face	2131.60	0.0
Topology	2178.68	0.0
Plastic Ocean	1347.75	$4.75\text{×}{10}^{-295}$
Pumpkin	2091.96	0.0
Scuba	2014.45	0.0
Beard	1233.01	$4.10\text{×}{10}^{-270}$
Lighten	0.04	0.85
Slimming	1.02	0.31
Eye Widening	0.94	0.33
Black and White	0.03	0.86
Glamorous	149.92	$1.80\text{×}{10}^{-34}$
Skin Eyes and Lips	0.56	0.45
Skin Smoothing	0.61	0.43
Nose	0.10	0.76
Sepia	30.90	2.71
Original images on baseline survey	87.56	$8.17\text{×}{10}^{-21}$

Table 2. Table 2 . Does this look strange? Statistical test results

Filter	Chi-Square Value	p-value
Flower Crown	469.94	$3.30\text{×}{10}^{-104}$
Fairy Lights	719.97	$1.36\text{×}{10}^{-158}$
Aviators	349.00	$6.99\text{×}{10}^{-78}$
Manly Face	693.52	$7.66\text{×}{10}^{-153}$
Topology	703.61	$4.90\text{×}{10}^{-155}$
Plastic Ocean	493.64	$2.32\text{×}{10}^{-109}$
Pumpkin	697.67	$9.58\text{×}{10}^{-154}$
Scuba	648.75	$4.16\text{×}{10}^{-143}$
Beard	493.33	$2.69\text{×}{10}^{-109}$
Lighten	1.28	0.26
Slimming	0.07	0.80
Eye Widening	0.0	1.0
Black and White	3.64	0.06
Glamorous	77.61	$1.26\text{×}{10}^{-18}$
Skin Eyes and Lips	3.50	0.06
Skin Smoothing	0.0	1.0
Nose	0.13	0.72
Sepia	14.59	0.00
Original images on baseline survey	1.36	0.24

Table 3. Table 3 . Does this look digitally altered? Statistical test results

Filter	Chi-Square Value	p-value
Flower Crown	272.13	$3.90\text{×}{10}^{-61}$
Fairy Lights	249.36	$3.59\text{×}{10}^{-56}$
Aviators	289.28	$7.14\text{×}{10}^{-65}$
Manly Face	203.43	$3.74\text{×}{10}^{-46}$
Topology	268.63	$2.25\text{×}{10}^{-60}$
Plastic Ocean	231.99	$2.20\text{×}{10}^{-52}$
Pumpkin	257.23	$6.88\text{×}{10}^{-58}$
Scuba	263.76	$2.60\text{×}{10}^{-59}$
Beard	219.73	$1.04\text{×}{10}^{-49}$
Lighten	1.75	0.19
Slimming	1.51	0.22
Eye Widening	0.02	0.89
Black and White	5.82	0.02
Glamorous	43.49	$4.26\text{×}{10}^{-11}$
Skin, eyes, and lips	4.55	0.03
Skin smoothing	3.82	0.05
Nose	0.46	0.50
Sepia	64.86	$8.05\text{×}{10}^{-16}$
Original images on baseline survey	5.78	0.02

Table 4. Table 4 . How familiar are you with this style of image? Statistical test results

Filter	Chi-Square Value	p-value
Flower Crown	483.69	2.26
Fairy Lights	848.41	$2.51\text{×}{10}^{-182}$
Aviators	370.69	$5.96\text{×}{10}^{-79}$
Manly Face	824.74	$3.36\text{×}{10}^{-177}$
Topology	837.86	$4.83\text{×}{10}^{-180}$
Plastic Ocean	619.42	$9.71\text{×}{10}^{-133}$
Pumpkin	865.80	$4.28\text{×}{10}^{-186}$
Scuba	678.35	$1.69\text{×}{10}^{-145}$
Beard	706.39	$1.44\text{×}{10}^{-151}$
Lighten	12.87	0.01
Slimming	3.58	0.47
Eye Widening	13.31	0.01
Black and White	47.32	$1.31\text{×}{10}^{-09}$
Glamorous	127.69	$1.21\text{×}{10}^{-26}$
Skin Eyes and Lips	4.24	0.37
Skin Smoothing	10.34	0.04
Nose	8.50	0.07
Sepia	134.17	$5.00\text{×}{10}^{-28}$
Original images on baseline survey	418.33	$3.05\text{×}{10}^{-89}$

Table 5. Table 5 . How strange does this look? Statistical test results

Filter	Chi-Square Value	p-value
Flower Crown	840.43	$1.34\text{×}{10}^{-180}$
Fairy Lights	995.28	$3.77\text{×}{10}^{-214}$
Aviators	689.72	$5.86\text{×}{10}^{-148}$
Manly Face	1019.36	$2.27\text{×}{10}^{-219}$
Topology	1051.14	$2.95\text{×}{10}^{-226}$
Plastic Ocean	859.83	$8.42\text{×}{10}^{-185}$
Pumpkin	1234.49	$5.31\text{×}{10}^{-266}$
Scuba	951.64	$1.08\text{×}{10}^{-204}$
Beard	791.64	$4.96\text{×}{10}^{-170}$
Lighten	1.45	0.84
Slimming	3.05	0.55
Eye Widening	0.49	0.97
Black and White	70.25	$2.00\text{×}{10}^{-14}$
Glamorous	220.78	$1.27\text{×}{10}^{-46}$
Skin, eyes, and lips	1.90	0.76
Skin smoothing	0.16	1.00
Nose	2.22	0.70
Sepia	162.43	$4.41\text{×}{10}^{-34}$
Original images on baseline survey	89.69	$1.53\text{×}{10}^{-18}$

Table 6. Table 6 . How digitally altered does this look? Statistical test results

Filter	Chi-Square Value	p-value
Flower Crown	948.42	$5.36\text{×}{10}^{-204}$
Fairy Lights	1119.82	$3.83\text{×}{10}^{-241}$
Aviators	873.64	$8.55\text{×}{10}^{-188}$
Manly Face	1120.81	$2.33\text{×}{10}^{-241}$
Topology	1224.20	$9.04\text{×}{10}^{-264}$
Plastic Ocean	999.81	$3.91\text{×}{10}^{-215}$
Pumpkin	1387.15	$4.23\text{×}{10}^{-299}$
Scuba	1210.85	$7.09\text{×}{10}^{-261}$
Beard	833.37	$4.53\text{×}{10}^{-179}$
Lighten	4.51	0.34
Slimming	12.02	0.02
Eye Widening	6.32	0.18
Black and White	127.61	$1.26\text{×}{10}^{-26}$
Glamorous	354.89	$1.54\text{×}{10}^{-75}$
Skin, eyes, and lips	19.41	0.00065
Skin smoothing	5.36	0.25
Nose	1.18	0.88
Sepia	251.69	$2.82\text{×}{10}^{-53}$
Original images on baseline survey	288.20	$3.80\text{×}{10}^{-61}$

Full text

Augmented Reality, Face Filter, Facial Perception, Social Media

††copyright: acmlicensed††journalyear: 2024††doi: XXXXXXX.XXXXXXX††ccs: Human-centered computing Empirical studies in HCI††ccs: Human-centered computing Mixed / augmented reality

1. INTRODUCTION

Facial perception is one of the most sophisticated human visual skills (Haxby et al., 2000). In addition to identifying individuals, the brain’s facial recognition abilities allow us to deduce “a wealth of information that facilitates social communication” (Haxby et al., 2000). One’s particular facial traits affect how they are perceived  (Oosterhof and Todorov, 2008). The judgments made from facial appearance, even if inaccurate, have powerful social impacts ranging from electoral success (Todorov et al., 2005) (Ballew and Todorov, 2007) (Little et al., 2007) to sentencing decisions (Blair et al., 2004) (Eberhardt et al., 2006).

Computer vision technologies allow us to modify the way our faces look online. As Adobe Photoshop and similar photo-editing tools have evolved over the past thirty years, computer users have been able to edit their faces by changing the image colors, smoothing their skin or whitening their teeth, adding makeup, and even adjusting the size and shape of various facial features. In contrast to these traditional photo editing techniques, more recently, Augmented Reality (AR) filters that overlay virtual content onto the image stream of a smartphone camera have emerged. These filters might add virtual sunglasses to a face or transform one’s likeness into that of a dog. Compared to traditional photo editing techniques that quietly modify reality to make it appear more beautiful but still plausible, many (but not all) AR filters intentionally change the image in unrealistic ways, transporting us to new worlds. In 2015, Snapchat was the first social media application to incorporate AR filters into their platform (INDE, 2020). Now, an increasing number of social media platforms including Instagram and TikTok offer AR filters to their users. Filtered photos (using both traditional and AR techniques) now drive a significant amount of engagement on social media (Lavrence and Cambre, 2020). One recent survey of 18-30 year old young women in the UK found that 90% of respondents had used such filters (Gill, 2021). AR beautification filters in both the video conferencing application Zoom and the Chinese mobile payment application Alipay allow users to view and present their faces through beauty filters (Shein, 2021) (Pitcher et al., 2020) (Peng, 2020).

How does the advent of facial editing on the Internet affect how we perceive human faces in general? One way humans determine facial attractiveness is based on what looks average to them, and according to Rhodes et al. (Rhodes et al., 2003) “exposure to systematically distorted faces produces corresponding changes in what looks average and what looks attractive.” Is online facial editing and the use of augmented reality filters changing what looks normal to us? As apps like Zoom that filter our faces in a real-time stream rather than just for a snapshot become more widespread, will this negatively or positively affect the social perceptions we draw from facial features?

In the early 2000s, before social media was as ubiquitous as it is today, concern was raised over the prevalence of photoshopped images in magazines (Wilson, 2009). Now, no longer limited to celebrity images, the trend of sharing modified selfies on social media has garnered media attention for its association with lower self-esteem and body dysmorphia (Shein, 2021) (Fastoso et al., 2021), sometimes referred to as Snapchat Dysmorphia (Eshiet, 2020) for its association with AR beautification filters on Snapchat. In extreme cases, body dysmorphia from beauty filter usage has pushed people towards plastic surgery (Hunt, 2019). However, some people experience positive effects from sharing filtered images. These include overcoming shyness, mood improvement, and more free self-expression (Pitcher et al., 2020) (Ryan-Mosley, 2021).

As the tech industry moves towards metaverse experiences, AR usage could become increasingly widespread. A better understanding of our perception of augmented images will help us both better care for the mental health of current AR users and build the metaverse responsibly. This paper presents a first step towards understanding how filtered faces are perceived. We approach this task with three research questions. Our key question is:

RQ1: Do filtered faces look normal or strange to people? Does this vary by which filter is used?

Responses to this question could be influenced by familiarity with that style of image filter and ability to tell that a filter has been applied. Thus, we additionally consider the following two questions:

RQ2: Which styles of facial filters look familiar to people?

RQ3: Can people tell that filtered images are digitally altered in the first place?

In this collaboration between computer scientists and psychologists, we present the results of six surveys: one two-alternative forced choice (2AFC) survey and one Likert scale survey for each of the three research questions above. Our results provide insight into how faces filtered with different styles of filters are being perceived. From these insights, we discuss how to design facial filters in socially responsible ways.

2. RELATED WORK

There is a wide body of literature on how AR filtering affects self-perception. The way one presents oneself online is strongly linked to one’s self-identity (Jin, 2012). AR beautification has the potential to increase self-esteem (Javornik and Pizzetti, 2017), though in some cases, AR may only change the perception of self when the user is dissatisfied with their appearance (Yim and Park, 2019). High self-esteem individuals may even resist this augmented self because it increases the gap between their ideal attractiveness and actual attractiveness, while low self-esteem individuals may welcome the augmented self (Javornik et al., 2021). Beauty filter users may identify with the filtered version of themselves more than with the real-life version of themselves (Ryan-Mosley, 2021).

A psychological study on self-face perception found that people had trouble discriminating between an unmodified picture of their face and a picture of their face with smaller features. Additionally, around a quarter of the study participants preferred pictures of themselves with larger eyes and around half of the participants preferred pictures of themselves with smaller noses (Felisberti and Musholt, 2014). Taken together this could indicate that AR filter users may have trouble distinguishing real photos of themselves from photos where their facial features are modified to a preferred size. Another study of self-face perception looked at the perception of personality, intelligence, emotion and appeal from facial traits in both unmodified faces and faces modified with AR. Contrary to the previous study’s findings, these researchers found that participants thought they looked significantly different with filters versus without (Fribourg et al., 2021) filters, even when the change to their face was minor. However, they also found that facial feature modification changed users’ perceptions of the personality traits their modified faces conveyed (Fribourg et al., 2021).

Other recent literature calls for the exploration of how AR affects the user’s relationship with the physical world (Laghari et al., 2021). However, little work has been done to address this call. In this study, we aim to respond to this call by studying if these augmented faces are becoming a new normal.

3. METHODS

We created six Institutional Review Board (IRB)-approved surveys to better understand how people perceive filtered faces. Details of these surveys follow.

Image Dataset. We created a dataset of filtered images and their original, unfiltered counterparts. 485 images of faces of unique subjects were selected from a publicly-available dataset of face images taken at the University of Notre Dame. We chose this dataset as the source of our images because it is one of the standard datasets of controlled facial images used across human biometrics research, including in the NIST evaluations of facial recognition models (Phillips et al., 2005) (Phillips et al., 2007). All images were taken indoors and the subjects were instructed to have a blank stare expression. The images were captured with a 4 Megapixel Canon PowerShot G2 camera. The dataset included 301 males and 184 females. The race breakdown was 345 White, 80 Asian, 2 Middle Eastern, 12 Hispanic, 8 Black, 14 South Asian and 24 unknown.

We filtered each of these 485 images with two different sets of facial filters: traditional and augmented reality. We distinguish between these styles of facial filtering because of their unique styles: traditional facial filters change the color tones of the image or beautify the face, while augmented reality filters overlay digital objects to the face or add a virtual mask. We acknowledge that in the wild there are many AR filters that are similar to traditional filters, modifying facial structure or applying makeup. Because of this overlap, we chose to focus on AR filters that are more distinct from traditional filters. We also compare these different filtering techniques because while the traditional filters have been around for upwards of thirty years, the augmented reality filters have been around fewer than ten. Distinguishing between the two enables us to consider if the longevity of a filtering technique impacts perception of faces edited in that style. For the traditional filters, we chose to use PortraitPro Studio Max, a photo editing software similar to Photoshop that offered convenience due to its ability to batch-apply face filters to large numbers of photos at once. Our AR filters were applied from the DeepAR free filter pack. We applied these to the images using a program we wrote using the DeepAR SDK. All of the PortraitPro and DeepAR filters used can also be seen in Figure 2.

Survey Questions. We wanted to know if filtered images looked normal or strange to the survey participants (RQ1). However, we also wanted to assess if filters determined to be strange-looking have an inherent strange-ness to them or if the strange-ness stems from unfamiliarity with the filter (RQ2). Additionally, because some of the filters produce realistic-looking images, do people even realize they are looking at a filtered image (RQ3)? We explored these questions with the following two-alternative forced choice (2AFC) and Likert scale phrasings:

• •

Are you familiar with this style of image? Response options: Yes, No

• •

Does this image look strange? Response options: Yes, No

• •

Does the image of this person look digitally altered? Response options: Yes, No

• •

On a scale of 1 to 5, how similar is this style of image to what you’ve seen before? 1: This is completely unlike any style of photo I’ve seen before. 5: This is very similar to styles of photos I’ve seen before

• •

On a scale of 1 to 5, how strange does this image look? 1: Not strange at all. 5: Very strange

• •

On a scale of 1 to 5, how digitally altered does the image of this person look? 1: Not digitally altered at all. 5: Heavily digitally altered

These surveys are modeled after surveys for studying human facial perception in social psychology. Studies on the other race effect (O’toole et al., 1994), facial attractiveness (Rhodes et al., 2003), and judgements of personality traits from facial appearance have used Likert scale surveys to measure human evaluation of faces (Willis and Todorov, 2006) (Todorov et al., 2009). Previous studies of human perception of virtual and augmented reality faces have also employed Likert scale surveys to assess perceived traits from the faces (Ferstl and McDonnell, 2018) (Javornik and Pizzetti, 2017) (Fribourg et al., 2021). This study also draws inspiration from Likert scale surveys in the field of biometrics, including collecting first impressions of faces to train a neural network to predict how crowds assign social attributes to faces (McCurrie et al., 2017) and comparing human and algorithmic abilities to detect facial retouching (Bharati et al., 2022).

Each of the six surveys was taken by a different subset of participants so that a participant’s experience with one survey wouldn’t bias their responses in a future survey. Consequently, we cannot claim causality from the survey results (e.g. we cannot not claim that images filtered with a particular filter are viewed as strange because the filter is unfamiliar). However, considering the results of the surveys in tandem can still provide insight into how these images are being perceived. Additionally, the use of six different survey questions provided us with multiple views into how filtered faces are being perceived in comparison to unedited faces.

We performed pilot studies to validate the phrasings of the survey questions, iterating on different phrasings if needed. If for a particular survey the first question phrasing elicited responses with no clear pattern then we iterated on the question phrasings until we found clear patterns elicited in the responses.

Survey Logistics. The surveys were hosted on Amazon Mechanical Turk in order to easily recruit a large sample of participants. Each survey asked only one of the above questions. Participants were limited to taking only one of the six surveys. The surveys were restricted to participants in the United States. Each participant was asked to answer that survey’s question for a random selection of 100 images from the dataset of original, traditionally-filtered, and AR-filtered images. A fixation cross was shown in between images. The survey took approximately 10 minutes to complete and participants were compensated 1 USD. Enough participants were recruited for each survey such that each image was seen at least 5 times: with the size of our dataset, this meant that at least 461 participants were required per survey. The total number of participants per survey was 517 for Are you familiar with this style of image?, 477 for Does this image look strange?, 479 for Does the image of this person look digitally altered?, 503 for How similar is this style of image to what you’ve seen before?, 464 for How strange does this image look?, and 465 for How digitally altered does the image of this person look? Users responded to the survey by pressing the ‘F’ key on their keyboard for yes and the ‘J’ key for no in the case of the 2AFC surveys, and by pressing a number from ‘1’ to ‘5’ on their keyboard for the Likert surveys. Each survey contained 5 control questions interspersed amongst the other survey questions in which the users were asked to hit the letter ‘A’ on their keyboard. The control questions were meant to catch users who repetitively hit keys to submit answers without paying attention to the questions. Participants had to pass 3 of the 5 control questions in order for their survey responses to be accepted. Demographic data regarding age, gender identity, and social media usage was collected.

Baseline Surveys. We additionally conducted baseline surveys in which the same survey questions were asked but only the original images were shown. The purpose of the baseline surveys was to assess how the original images are perceived without the influence or bias of the filtered images mixed into the dataset. Each baseline survey had between 25 and 38 participants. Results of baseline surveys are included in the appendix.

4. RESULTS

For each survey, we examined the results for the original images, traditionally edited images, and AR-edited images. Results are visualized by the histograms in Figures 3-6. Additionally, for each survey we performed Chi-squared tests comparing the distributions of responses from the original images and filtered images, for each of the filter types.

The total recorded amounts of each response option per filter type and per survey, as well as full results of the Chi-squared tests including p-values can be found in the appendix. Although demographic data, including age, gender, and social media usage, was collected, it surprisingly did not provide any meaningful results and is omitted for brevity.

Traditional Filter Two-Alternative Forced Choice Results As with the original images, all traditionally-filtered images were more frequently rated as familiar, not strange, and not digitally altered. However, the Sepia and Glamorous filters had closer results on all surveys compared to the originals, leading to a significant difference (p $<$ 0.05). On the digital alteration survey, the filters Black and White, Skin Smoothing, and Skin Eyes and Lips also elicited significantly different (p $<$ 0.05) responses to the originals due to their closer results.

Traditional Filter Likert Results All of the traditional filters followed similar response distributions to the responses to the originals. On the familiarity survey, both the originals and all traditional filters had mostly “Very Similar” (‘5’) responses followed by “Completely Unlike” (‘1’) responses. In most cases this was followed by ‘4,’ then ‘3,’ then ‘2’ responses. On the strangeness survey, the originals and all traditional filters had mostly “Not Strange” (‘1’) responses followed by “Very Strange” (‘5’) responses. The originals and most traditional filters then had more ‘2’ responses, followed by ‘3’ and ‘4 responses. More filters were significantly different from the originals compared to the 2AFC surveys, especially Black and White, Sepia, and Glamorous, which were significantly different (p $<$ 0.05) in all three surveys. We acknowledge that significant difference in response distributions is more likely on Likert surveys even across response distributions that appear similar due to the higher number of response options.

AR Filter Two-Alternative Forced Choice and Likert Results On all 2AFC and Likert surveys, every AR filter elicited significantly different results (p $<$ 0.05) from the originals. Moreover, with one exception, on every survey, every AR filter produced an almost inverse distribution of responses from the responses to the originals. In other words, when the originals elicited predominantly ‘yes’ responses on the 2AFC surveys, the AR-filtered faces elicited predominantly ‘no’ responses, and vice versa. On the Likert surveys, the frequency of ‘5’ responses for the original images would resemble the frequency of ‘1’ responses for the AR-filtered images and vice versa. A similar pattern occurred for the ‘2,’ ‘3,’ and ‘4’ responses across the original images and AR-filtered images. The one exception to this is the Aviators filter, which on the familiarity 2AFC survey was more frequently marked as “familiar” than “unfamiliar,” like the originals. However, the Aviators filter only had slightly more “familiar” than “unfamiliar” responses, while the originals had many more “familiar” than “unfamiliar” responses.

5. DISCUSSION AND CONCLUSION

The results of our six surveys show that faces filtered with traditional filters are perceived similarly to unedited faces, whereas faces filtered with AR filters are perceived differently from unedited faces. In most cases, the responses to faces filtered with traditional filters are not significantly different from the responses to unedited faces. Even when the responses are significantly different, the distribution of responses to traditionally-filtered faces is relatively similar to the distribution of responses to unedited faces. On the other hand, responses to faces filtered with AR filters are not only significantly different from the responses to unedited faces but have a completely different shaped distribution of responses. In many cases the response distribution for the AR filtered faces is almost the exact inverse of the response distribution for the original and traditionally-edited faces. That these distributions are inverses of each other is notable: people respond to AR-filtered faces opposite from the way that they respond to unedited or traditionally-filtered faces.

Thus it appears that the more traditionally edited faces have become a “new normal” in human facial perception, whereas AR-filtered faces have not. This is especially seen in the cases of the Black and White, Skin Eyes and Lips, Slimming, and Skin Smoothing filters which were all identified as digitally altered at somewhat higher (and significantly different) rates than the originals, but on the familiarity and strangeness surveys were not perceived significantly differently from the originals. Because these images are more clearly filtered but still appear normal, our research indicates that these filters in particular are being incorporated into a new normal of what a face should look like.

Why are traditionally-filtered faces perceived similarly to unedited faces but AR-filtered faces are not? One possible explanation is the difference in length of time that these filters have existed. Sepia and Black and White images were created with the first cameras and pre-date computer filters. The other traditional filters we selected are in the style of the photo editing that proliferated with Photoshop beginning thirty years ago and was ubiquitous in magazines before the spread of social media. On the other hand, AR filters have only been around for 10 years. Given that the traditional filtering techniques have existed for much longer than the AR filters have, is it just a matter of time until we view AR-filtered images as normal? We recommend future work in continuing to monitor perception of AR-filtered faces over time, to determine if a shift to familiarity will occur.

Another possible explanation for the difference in perception is a difference in the purpose of the filter and the style of its modifications. With the exception of the Black and White, Sepia, and Lighten filters, the traditional filters we used in this study are beautification filters, which serve to enhance the appearance of faces existing in the real world. On the other hand, AR filters can help construct alternate realities and stimulate the imagination. With this in mind, our results indicate that these different types of filters are working as they were intended. The faces filtered with the traditional beautification filters still appear normal, allowing them to appear as real-world faces. Conversely, even the most realistic-looking AR filters in our study still caused faces to be perceived as strange, but perhaps this perceived strangeness indicates that the filters are allowing us to escape from reality.

Although both types of filters are working as intended, we must ask: what are the implications of these filters working as intended? Do we want faces filtered with beautification filters to be perceived as normal when those faces don’t actually exist in the real world as they appear in the image? An acne-prone teenager would surely experience a boost of self-confidence with the ability to smooth out their skin before posting a selfie on social media. On the other hand, the Snapchat Dysphoria phenomenon indicates that there is a danger in filtering reality and presenting the results as real. Because the AR-filtered faces in our study were perceived so diametrically differently from unedited faces, it may actually be safer to use these types of filters. They won’t confound our facial perception and cause phenomena like Snapchat Dysphoria in the way that traditional beautification filters would.

One caveat in our support for AR filters is the use of AR beautification filters. We limited the AR filters we used in our study to those that overlay digital objects (Flower Crown, Fairy Lights, Aviators, Plastic Ocean, Pumpkin, Scuba, and Beard) or those that apply unrealistic-looking masks (Manly Face and Topology) to the faces. We did not include AR beautification filters in our study due to their resemblance to the traditional filters we used. However, AR beautification filters take photo editing techniques previously limited to still images and enable them to be applied to a face in real time. We anticipate both the pros and cons of traditional beautification filtering will be amplified when our faces can be filtered in real time. As mentioned in the introduction, this is already possible on Zoom and Alipay, and we encourage future work to study user experiences with real-time beauty filtering on these apps.

Based on our results we recommend AR/VR designers focus more on creating filters that overlay digital objects, transform the face in unrealistic ways, or otherwise promote creativity. We believe that the difference in perception of faces with these filters compared to unfiltered faces indicates that these filters allow us to enter more imaginative worlds. On the other hand, AR beautification filters may continue to further the social concerns already raised by traditional photo beautification techniques. This recommendation is consistent with other literature on AR filter usage. Javornik et. al. found that different motivations for AR filter usage had different effects on well-being: if AR filter usage is driven by a desire to change one’s image to fit an idealized version of oneself that is unattainable offline, then AR filter usage reduces self-acceptance. However, if the AR filter usage aligns with one’s already-existing self image, or helps one actively explore or transform their self-image, it can increase self-acceptance (Javornik et al., 2022). Additionally, the MIT Technology Review reported that while there are many cases of beautification filters having negative effects on mental health, artistic or creative filters can have a positive effect on mental health (Ryan-Mosley, 2021).

One limitation of our study is that our preexisting dataset of faces contained a much higher representation of light skinned faces versus darker complexions. Another limitation is that some of the filters confound with real-world facial modifications, such as applying real makeup or real sunglasses. In fact, we see from our results that of the AR filters, the Aviators filter deviated the most from the results of the other AR filters. Is there a difference in response to faces with real sunglasses or makeup versus faces with virtual sunglasses or makeup? We encourage future work to build on this study by exploring these limitations. We also recommend future work performing the same studies using filters found on popular social media platforms such as Snapchat and Instagram. These specific filters are more ubiquitous than the ones we used in our study, and thus would provide a useful point of comparison.

Conclusion. We presented the results of six surveys that measure perceived familiarity, strangeness, and digital alteration of filtered faces. We demonstrated that faces with traditional filters applied are generally perceived similarly to unmodified faces as familiar, not strange, and not digitally altered, but AR-filtered faces are perceived as not familiar, strange, and digitally altered. We hope that this work can help inspire conversation on social responsibility in facial filter usage. We must continue to ask: Are AR filters actually more safe to use than traditional filters because they take us out of reality rather than try to perfect it? Or will these AR filters, with time, join the more traditional edits in our perception of what is normal? If so, what are the implications of this? While we should continue to study whether AR filters will be perceived as “normal” over time, we recommend for now that AR/VR researchers note that promoting imagination and creativity may be healthier than trying to filter reality.

Acknowledgements.

The authors would like to thank Annalisa Szymanski and Daniel Gonzalez Cedre for their feedback on this work.

Appendix A Baseline Survey Results

“Are you familiar with this style of image?,” “Does this image look strange?,” and “How strange does this image look?” had 25 survey participants, “Does the image of this person look digitally altered?” and “How digitally altered does this image look?” had 26 participants, and “How similar is this to what you’ve seen before?” had 38 participants.

Are you familiar with this style of image? Results. In the baseline study, the original images had around 72.5% ‘yes’ responses and 27.5% ‘no’ responses. The original images in the context of the full study had around 83.4% ‘yes’ responses and 16.6% ‘no’ responses. These results are significantly different from each other (p $<$ 0.05) despite that on both surveys many more respondents responded ‘yes’ than ‘no.’ With the filtered images mixed in to the dataset, the original images are rated as more familiar. We conjecture that when there are no filtered images mixed in, subjects rate fewer of the unfiltered images as familiar because the question primes them to think there are unfamiliar images in the set. However, the converse could be true: that the presence of less familiar filtered images caused subjects to find the unfiltered images to be more familiar.

Does this image look strange? Results. In the baseline study, the original images had 30% ‘yes’ responses and 70% ‘no’ responses. In the full study, the original images had 31.6% ‘yes’ responses and 68.4% ‘no’ responses. These two sets of responses were not found to be significantly different from each other on a Chi-Squared test.

Does this image look digitally altered? Results. In the baseline study, the original images had around 35.8% ‘yes’ and 64.2% ‘no’ responses. In the context of the full study, the original images had 39.2% ‘yes’ responses and 60.8% ‘no’ responses. This is significantly different (p $<$ 0.05) from the responses to the original images in the context of the full survey. Although the results were significantly different, they were similar. However, the presence of clearly filtered images in the dataset caused respondents to think that slightly more unedited images were edited.

How familiar does this style of image look? Results. In the baseline study, the original images had around 17% ‘1’, 14% ‘2’, 16% ‘3’, 19% ‘4’, and 33% ‘5’. In the full study, the original responses followed a distribution of around 23-26% ‘1’, around 5-6% ‘2’, 8-10% ‘3’, around 11% ‘4’, and around 45-50% ‘5’. The baseline survey produced responses significantly different from the originals (p $<$ 0.05) on a Chi-Squared test. In this case, the main difference in responses came from more of the ‘2’, ‘3’, and ‘4’, responses being selected in the baseline study. We speculate that the presence of filtered images in the dataset caused respondents to select the response options at the extremes, because the different filters caused the images to all look very different from each other. In the baseline survey where only unfiltered images were included, because the images were all more relatively similar to each other, that could have encouraged the participants to use more of the response options in the middle.

How strange does this image look? Results. The original image responses in the main study were 55% ‘1’, 11% ‘2’, 8% ‘3’, 7% ‘4’, and 19% ‘5’. In the baseline study, the original images had around 44% ‘1’, 11% ‘2’, 11% ‘3’, 13% ‘4’, and 22% ‘5’ responses. The distribution is significantly different from the response distribution for the original images in the main study. We observe two main differences between the response distributions. First, the baseline study had both more ‘2,’ ‘3,’ and ‘4,’ responses compared to the responses to the originals in the full study, and the responses skewed more towards looking more strange. Second, when there were no filtered images in the dataset, users responded that the original images looked more strange. This is similar to what we saw in the similarity surveys, where participants selected more of the ‘2’, ‘3’, and ‘4’ options on the baseline survey than on the full survey, and participants answered “Yes” or ‘5’ for more unfiltered images on the baseline study than in the full study. We propose the same explanation for these differences as we did with the similarity survey.

How digitally altered does this image look? Results. In the main study, the original images had 45-53% ‘1’ responses, 14-15% ‘2’ responses, 6-11% ‘3’, and ‘4’ responses, and 18% ‘5’ responses. In the baseline study, the original images had around 39% ‘1’, 23% ‘2’, 18% ‘3’, 9% ‘4’, and 11% ‘5’ responses. This response distribution was significantly different (p $<$ 0.05) from the distribution of responses for the original images in the full study. The main difference is more ‘2,’ ‘3,’ and ‘4,’ responses, especially ‘2’ responses, in the baseline study. As mentioned earlier, we see this trend in the other two baseline surveys, and we speculate that when filtered images are present, the more extreme differences in the appearance of the images encouraged participants to choose the more extreme options on the Likert scale.

Appendix B Full Survey Results

In this section we provide the number of each type of response to each survey, broken down by filter.

B.1. Are you familiar with this style of image? Results

Original: Yes: $n=2125$, No: $n=422$

PortraitPro: Yes: $n=18312$, No: $n=4293$

DeepAR: Yes: $n=6385$, No: $n=16288$

Flower Crown: Yes: $n=1097$, No: $n=1406$

Fairy Lights: Yes: $n=386$, No: $n=2140$

Aviators: Yes: $n=1284$, No: $n=1223$

Manly Face: Yes: $n=465$, No: $n=2048$

Topology: Yes: $n=453$, No: $n=2078$

Plastic Ocean: Yes: $n=815$, No: $n=1696$

Pumpkin: Yes: $n=484$, No: $n=2039$

Scuba: Yes: $n=520$, No: $n=2017$

Beard: Yes: $n=881$, No: $n=1641$

Lighten: Yes: $n=2094$, No: $n=423$

Slim Male: Yes: $n=2058$, No: $n=442$

Eye Widening: Yes: $n=2127$, No: $n=391$

Black and White: Yes: $n=2086$, No: $n=421$

Glamorous: Yes: $n=1716$, No: $n=782$

Skin eyes and lips: Yes: $n=2089$, No: $n=440$

Skin Smoothing: Yes: $n=2107$, No: $n=393$

Nose: Yes: $n=2090$, No: $n=426$

Sepia: Yes: $n=1945$, No: $n=575$

Original images on the baseline survey: Yes: $n=1812$, No: $n=688$

B.2. Does this image look strange? Results

Original: Yes: $n=797$, No: $n=1725$

PortraitPro: Yes: $n=7761$, No: $n=14832$

DeepAR: Yes: $n=14795$, No: $n=7795$

Flower Crown: Yes: $n=1563$, No: $n=953$

Fairy Lights: Yes: $n=1734$, No: $n=760$

Aviators: Yes: $n=1452$, No: $n=1059$

Manly Face: Yes: $n=1729$, No: $n=786$

Topology: Yes: $n=1728$, No: $n=774$

Plastic Ocean: Yes: $n=1583$, No: $n=934$

Pumpkin: Yes: $n=1730$, No: $n=782$

Scuba: Yes: $n=1696$, No: $n=815$

Beard: Yes: $n=1580$, No: $n=932$

Lighten: Yes: $n=831$, No: $n=1677$

Slim Male: Yes: $n=802$, No: $n=1706$

Eye Widening: Yes: $n=790$, No: $n=1709$

Black and White: Yes: $n=856$, No: $n=1649$

Glamorous: Yes: $n=1095$, No: $n=1412$

Skin eyes and lips: Yes: $n=858$, No: $n=1657$

Skin Smoothing: Yes: $n=798$, No: $n=1725$

Nose: Yes: $n=809$, No: $n=1710$

Sepia: Yes: $n=922$, No: $n=1587$

Original images on the baseline survey: Yes: $n=751$, No: $n=1749$

B.3. Does this image look digitally altered? Results

Original: Yes: $n=982$, No: $n=1525$

PortraitPro: Yes: $n=9701$, No: $n=12850$

DeepAR: Yes: $n=13872$, No: $n=8670$

Flower Crown: Yes: $n=1563$, No: $n=937$

Fairy Lights: Yes: $n=1532$, No: $n=957$

Aviators: Yes: $n=1589$, No: $n=925$

Manly Face: Yes: $n=1489$, No: $n=1020$

Topology: Yes: $n=1557$, No: $n=939$

Plastic Ocean: Yes: $n=1530$, No: $n=991$

Pumpkin: Yes: $n=1548$, No: $n=954$

Scuba: Yes: $n=1554$, No: $n=946$

Beard: Yes: $n=1510$, No: $n=1001$

Lighten: Yes: $n=1024$, No: $n=1471$

Slim Male: Yes: $n=1024$, No: $n=1479$

Eye Widening: Yes: $n=985$, No: $n=1515$

Black and White: Yes: $n=1062$, No: $n=1433$

Glamorous: Yes: $n=1217$, No: $n=1295$

Skin eyes and lips: Yes: $n=1063$, No: $n=1458$

Skin Smoothing: Yes: $n=1053$, No: $n=1459$

Nose: Yes: $n=1004$, No: $n=1497$

Sepia: Yes: $n=1269$, No: $n=1243$

Original images on the baseline survey: Yes: $n=896$, No: $n=1604$

B.4. How familiar does this style of image look? Results

Responses are in a Likert scale from ”1: completely unfamiliar” to ”5: very similar.”

Original: 1: $n=610$, 2: $n=146$, 3: $n=211$, 4: $n=289$, 5: $n=1320$

PortraitPro: 1: $n=6231$, 2: $n=1453$, 3: $n=2397$, 4: $n=2731$, 5: $n=10406$

DeepAR: 1: $n=11713$, 2: $n=2564$, 3: $n=2613$, 4: $n=1162$, 5: $n=5175$

Flower Crown: 1: $n=1058$, 2: $n=341$, 3: $n=344$, 4: $n=167$, 5: $n=659$

Fairy Lights: 1: $n=1416$, 2: $n=308$, 3: $n=257$, 4: $n=94$, 5: $n=503$

Aviators: 1: $n=957$, 2: $n=326$, 3: $n=354$, 4: $n=241$, 5: $n=708$

Manly Face: 1: $n=1550$, 2: $n=199$, 3: $n=187$, 4: $n=116$, 5: $n=535$

Topology: 1: $n=1488$, 2: $n=218$, 3: $n=248$, 4: $n=104$, 5: $n=503$

Plastic Ocean: 1: $n=1184$, 2: $n=334$, 3: $n=333$, 4: $n=143$, 5: $n=583$

Pumpkin: 1: $n=1532$, 2: $n=194$, 3: $n=252$, 4: $n=72$, 5: $n=539$

Scuba: 1: $n=1256$, 2: $n=291$, 3: $n=351$, 4: $n=100$, 5: $n=589$

Beard: 1: $n=1272$, 2: $n=353$, 3: $n=287$, 4: $n=125$, 5: $n=556$

Lighten: 1: $n=697$, 2: $n=134$, 3: $n=251$, 4: $n=286$, 5: $n=1230$

Slim Male: 1: $n=630$, 2: $n=149$, 3: $n=235$, 4: $n=307$, 5: $n=1260$

Eye Widening: 1: $n=660$, 2: $n=154$, 3: $n=263$, 4: $n=286$, 5: $n=1203$

Black and White: 1: $n=689$, 2: $n=151$, 3: $n=257$, 4: $n=389$, 5: $n=1084$

Glamorous: 1: $n=797$, 2: $n=241$, 3: $n=317$, 4: $n=269$, 5: $n=958$

Skin eyes and lips: 1: $n=669$, 2: $n=139$, 3: $n=217$, 4: $n=290$, 5: $n=1263$

Skin Smoothing: 1: $n=686$, 2: $n=146$, 3: $n=244$, 4: $n=261$, 5: $n=1247$

Nose: 1: $n=645$, 2: $n=156$, 3: $n=256$, 4: $n=285$, 5: $n=1235$

Sepia: 1: $n=758$, 2: $n=183$, 3: $n=357$, 4: $n=358$, 5: $n=926$

Original images on the baseline survey: 1: $n=675$, 2: $n=588$, 3: $n=663$, 4: $n=749$, 5: $n=1325$

B.5. How strange does this image look? Results

Responses are in a Likert scale from ”1: not strange at all” to ”5: very strange.” Original: 1: $n=1345$, 2: $n=269$, 3: $n=186$, 4: $n=178$, 5: $n=463$

PortraitPro: 1: $n=10991$, 2: $n=2886$, 3: $n=2017$, 4: $n=1792$, 5: $n=4310$

DeepAR: 1: $n=3921$, 2: $n=2009$, 3: $n=2195$, 4: $n=2532$, 5: $n=11316$

Flower Crown: 1: $n=396$, 2: $n=323$, 3: $n=433$, 4: $n=340$, 5: $n=952$

Fairy Lights: 1: $n=407$, 2: $n=202$, 3: $n=162$, 4: $n=289$, 5: $n=1379$

Aviators: 1: $n=466$, 2: $n=441$, 3: $n=433$, 4: $n=304$, 5: $n=800$

Manly Face: 1: $n=465$, 2: $n=142$, 3: $n=143$, 4: $n=199$, 5: $n=1497$

Topology: 1: $n=415$, 2: $n=174$, 3: $n=136$, 4: $n=258$, 5: $n=1460$

Plastic Ocean: 1: $n=420$, 2: $n=215$, 3: $n=343$, 4: $n=413$, 5: $n=1052$

Pumpkin: 1: $n=467$, 2: $n=89$, 3: $n=92$, 4: $n=129$, 5: $n=1665$

Scuba: 1: $n=423$, 2: $n=203$, 3: $n=183$, 4: $n=259$, 5: $n=1367$

Beard: 1: $n=462$, 2: $n=220$, 3: $n=270$, 4: $n=341$, 5: $n=1144$

Lighten: 1: $n=1320$, 2: $n=281$, 3: $n=179$, 4: $n=177$, 5: $n=491$

Slim Male: 1: $n=1287$, 2: $n=277$, 3: $n=199$, 4: $n=180$, 5: $n=497$

Eye Widening: 1: $n=1370$, 2: $n=271$, 3: $n=181$, 4: $n=176$, 5: $n=450$

Black and White: 1: $n=1133$, 2: $n=436$, 3: $n=218$, 4: $n=236$, 5: $n=422$

Glamorous: 1: $n=853$, 2: $n=337$, 3: $n=382$, 4: $n=255$, 5: $n=616$

Skin eyes and lips: 1: $n=1355$, 2: $n=270$, 3: $n=192$, 4: $n=194$, 5: $n=432$

Skin Smoothing: 1: $n=1346$, 2: $n=276$, 3: $n=187$, 4: $n=173$, 5: $n=465$

Nose: 1: $n=1377$, 2: $n=246$, 3: $n=195$, 4: $n=164$, 5: $n=458$

Sepia: 1: $n=950$, 2: $n=492$, 3: $n=284$, 4: $n=237$, 5: $n=479$

Original images on the baseline survey: 1: $n=1092$, 2: $n=269$, 3: $n=278$, 4: $n=318$, 5: $n=543$

B.6. How digitally altered does this image look? Results

Responses are in a Likert scale from ”1: not digitally altered” to ”5: heavily digitally altered.” Original: 1: $n=1297$, 2: $n=357$, 3: $n=179$, 4: $n=144$, 5: $n=451$

PortraitPro: 1: $n=9952$, 2: $n=3666$, 3: $n=2305$, 4: $n=1796$, 5: $n=4118$

DeepAR: 1: $n=3871$, 2: $n=1468$, 3: $n=1662$, 4: $n=2971$, 5: $n=11867$

Flower Crown: 1: $n=373$, 2: $n=265$, 3: $n=290$, 4: $n=464$, 5: $n=1034$

Fairy Lights: 1: $n=405$, 2: $n=138$, 3: $n=156$, 4: $n=363$, 5: $n=1364$

Aviators: 1: $n=377$, 2: $n=339$, 3: $n=408$, 4: $n=438$, 5: $n=864$

Manly Face: 1: $n=474$, 2: $n=91$, 3: $n=133$, 4: $n=258$, 5: $n=1471$

Topology: 1: $n=460$, 2: $n=89$, 3: $n=79$, 4: $n=263$, 5: $n=1534$

Plastic Ocean: 1: $n=423$, 2: $n=162$, 3: $n=214$, 4: $n=390$, 5: $n=1237$

Pumpkin: 1: $n=439$, 2: $n=89$, 3: $n=50$, 4: $n=142$, 5: $n=1705$

Scuba: 1: $n=437$, 2: $n=99$, 3: $n=105$, 4: $n=256$, 5: $n=1530$

Beard: 1: $n=483$, 2: $n=196$, 3: $n=227$, 4: $n=397$, 5: $n=1128$

Lighten: 1: $n=1232$, 2: $n=376$, 3: $n=201$, 4: $n=162$, 5: $n=455$

Slim Male: 1: $n=1226$, 2: $n=387$, 3: $n=238$, 4: $n=146$, 5: $n=431$

Eye Widening: 1: $n=1309$, 2: $n=334$, 3: $n=210$, 4: $n=163$, 5: $n=411$

Black and White: 1: $n=967$, 2: $n=567$, 3: $n=257$, 4: $n=221$, 5: $n=414$

Glamorous: 1: $n=722$, 2: $n=360$, 3: $n=410$, 4: $n=344$, 5: $n=590$

Skin eyes and lips: 1: $n=1187$, 2: $n=393$, 3: $n=236$, 4: $n=182$, 5: $n=429$

Skin Smoothing: 1: $n=1241$, 2: $n=377$, 3: $n=216$, 4: $n=148$, 5: $n=444$

Nose: 1: $n=1284$, 2: $n=360$, 3: $n=194$, 4: $n=152$, 5: $n=435$

Sepia: 1: $n=784$, 2: $n=512$, 3: $n=343$, 4: $n=278$, 5: $n=509$

Original images on the baseline survey: 1: $n=1001$, 2: $n=607$, 3: $n=477$, 4: $n=228$, 5: $n=287$

Appendix C Full Chi-Squared Test Results

In this section we provide the results of the Chi-squared tests for each survey, comparing the results by filter to the results for the original images.