Stacking Brick by Brick: Aligned Feature Isolation for Incremental Face Forgery Detection

TL;DR

This paper introduces an incremental face forgery detection method that preserves learned features and mitigates forgetting by aligning feature distributions using a novel sparse replay technique and a latent-space detector.

Contribution

It proposes aligned feature isolation with Sparse Uniform Replay and a Latent-space Incremental Detector to improve incremental face forgery detection performance.

Findings

Outperforms existing methods on a new comprehensive benchmark.

Effectively mitigates catastrophic forgetting in incremental learning.

Enhances detection of diverse and evolving face forgeries.

Abstract

The rapid advancement of face forgery techniques has introduced a growing variety of forgeries. Incremental Face Forgery Detection (IFFD), involving gradually adding new forgery data to fine-tune the previously trained model, has been introduced as a promising strategy to deal with evolving forgery methods. However, a naively trained IFFD model is prone to catastrophic forgetting when new forgeries are integrated, as treating all forgeries as a single ''Fake" class in the Real/Fake classification can cause different forgery types overriding one another, thereby resulting in the forgetting of unique characteristics from earlier tasks and limiting the model's effectiveness in learning forgery specificity and generality. In this paper, we propose to stack the latent feature distributions of previous and new tasks brick by brick, $\textit{i.e.}$, achieving $\textbf{aligned feature isolation}$. In this manner, we aim to preserve learned forgery information and accumulate new knowledge by minimizing distribution overriding, thereby mitigating catastrophic forgetting. To achieve this, we first introduce Sparse Uniform Replay (SUR) to obtain the representative subsets that could be treated as the uniformly sparse versions of the previous global distributions. We then propose a Latent-space Incremental Detector (LID) that leverages SUR data to isolate and align distributions. For evaluation, we construct a more advanced and comprehensive benchmark tailored for IFFD. The leading experimental results validate the superiority of our method.