AnonySIGN: Novel Human Appearance Synthesis for Sign Language Video Anonymisation

TL;DR

AnonySign is an innovative method that automatically anonymizes sign language videos by synthesizing realistic human appearances from pose data, preserving sign content while protecting signer identity.

Contribution

This paper introduces AnonySign, a novel automatic approach for visual anonymization of sign language videos using pose extraction and style-consistent image synthesis within a variational autoencoder framework.

Findings

Achieves realistic and anonymous sign language video synthesis.

Retains original sign language content after anonymization.

Outperforms existing methods in realism and privacy protection.

Abstract

The visual anonymisation of sign language data is an essential task to address privacy concerns raised by large-scale dataset collection. Previous anonymisation techniques have either significantly affected sign comprehension or required manual, labour-intensive work. In this paper, we formally introduce the task of Sign Language Video Anonymisation (SLVA) as an automatic method to anonymise the visual appearance of a sign language video whilst retaining the meaning of the original sign language sequence. To tackle SLVA, we propose AnonySign, a novel automatic approach for visual anonymisation of sign language data. We first extract pose information from the source video to remove the original signer appearance. We next generate a photo-realistic sign language video of a novel appearance from the pose sequence, using image-to-image translation methods in a conditional variational autoencoder framework. An approximate posterior style distribution is learnt, which can be sampled from to synthesise novel human appearances. In addition, we propose a novel \textit{style loss} that ensures style consistency in the anonymised sign language videos. We evaluate AnonySign for the SLVA task with extensive quantitative and qualitative experiments highlighting both realism and anonymity of our novel human appearance synthesis. In addition, we formalise an anonymity perceptual study as an evaluation criteria for the SLVA task and showcase that video anonymisation using AnonySign retains the original sign language content.