A Spatial-Frequency Aware Multi-Scale Fusion Network for Real-Time Deepfake Detection

TL;DR

This paper introduces SFMFNet, a lightweight multi-scale neural network that effectively detects deepfakes in real-time by combining spatial textures and frequency artifacts with efficient feature interaction mechanisms.

Contribution

The paper presents a novel spatial-frequency hybrid module and a token-selective cross attention mechanism to improve real-time deepfake detection efficiency and accuracy.

Findings

Achieves high accuracy with low computational cost

Demonstrates strong generalization across datasets

Balances detection performance and real-time efficiency

Abstract

With the rapid advancement of real-time deepfake generation techniques, forged content is becoming increasingly realistic and widespread across applications like video conferencing and social media. Although state-of-the-art detectors achieve high accuracy on standard benchmarks, their heavy computational cost hinders real-time deployment in practical applications. To address this, we propose the Spatial-Frequency Aware Multi-Scale Fusion Network (SFMFNet), a lightweight yet effective architecture for real-time deepfake detection. We design a spatial-frequency hybrid aware module that jointly leverages spatial textures and frequency artifacts through a gated mechanism, enhancing sensitivity to subtle manipulations. A token-selective cross attention mechanism enables efficient multi-level feature interaction, while a residual-enhanced blur pooling structure helps retain key semantic cues during downsampling. Experiments on several benchmark datasets show that SFMFNet achieves a favorable balance between accuracy and efficiency, with strong generalization and practical value for real-time applications.

Tables

Table 1. Table 1: Compares the performance of various detection methods within the same dataset.The best results are highlighted in bold.

Model	FF++	FF-DF	FF-F2F	FF-FS	FF-NT	Avg.
RegNet[32]	0.9806	0.9903	0.9845	0.9892	0.9715	0.9833
GoogLeNet[33]	0.9810	0.9927	0.9877	0.9884	0.9554	0.9810
AlexNet[24]	0.9415	0.9632	0.9623	0.9576	0.8828	0.9414
ResNet18[25]	0.9810	0.9904	0.9860	0.9884	0.9592	0.9810
CNN-Aug[28]	0.8493	0.9048	0.8788	0.9026	0.7313	0.8533
Xception[26]	0.9637	0.9799	0.9785	0.9833	0.9385	0.9687
Capsule[35]	0.8421	0.8669	0.8634	0.8734	0.7804	0.8452
FWA[34]	0.8765	0.9210	0.9000	0.8843	0.8120	0.8787
X-ray[29]	0.9592	0.9794	0.9872	0.9871	0.9290	0.9683
FFD[30]	0.9624	0.9803	0.9784	0.9853	0.9306	0.9674
UCF[31]	0.9705	0.9883	0.9840	0.9896	0.9441	0.9753
F3Net[27]	0.9635	0.9793	0.9796	0.9844	0.9354	0.9684
SPSL[36]	0.9610	0.9781	0.9754	0.9829	0.9299	0.9654
ours	0.9868	0.9949	0.9884	0.9890	0.9748	0.9867

Table 2. Table 2: Examines the model’s generalization ability across different datasets.

Model	FF++	DFD	CDF2	DFDCP	DFDC	UADFV	Avg.	FLOPs(G)	Params(M)
RegNet[32]	0.9806	0.8387	0.7173	0.7105	0.6902	0.7965	0.7889	2.11	9.19
GoogLeNet[33]	0.8321	0.7321	0.6967	0.7259	0.6505	0.8785	0.7526	1.96	13.01
AlexNet[24]	0.9415	0.7787	0.7538	0.7127	0.6918	0.8637	0.7903	1.32	61.12
ResNet18[25]	0.9810	0.8030	0.7731	0.7039	0.6736	0.8653	0.7999	2.37	11.69
CNN-Aug[28]	0.8493	0.6464	0.7027	0.6170	0.6361	0.8739	0.7209	1.93	11.73
Xception[26]	0.9637	0.8163	0.7365	0.7374	0.7077	0.9379	0.8165	5.98	20.81
Capsule[35]	0.8421	0.6841	0.7472	0.6568	0.6465	0.9078	0.7474	19.80	20.21
FWA[34]	0.8765	0.7403	0.6680	0.6373	0.6132	0.8539	0.7315	5.98	20.94
X-ray[29]	0.9592	0.7655	0.6786	0.6942	0.6326	0.8989	0.7715	8.93	96.50
FFD[30]	0.9624	0.8024	0.7435	0.7426	0.7029	0.9450	0.8164	6.01	22.37
UCF[31]	0.9705	0.8074	0.7527	0.7594	0.7179	0.9528	0.8267	10.53	47.64
F3Net[27]	0.9635	0.7975	0.7352	0.7354	0.7021	0.9347	0.8114	6.19	21.32
SPSL[36]	0.9610	0.8122	0.7650	0.7408	0.7040	0.9424	0.8209	7.65	20.82
ours	0.9868	0.8781	0.8111	0.8603	0.7307	0.9420	0.8682	1.27	6.64

Table 3. Table 3: Ablation study of each module on CDF2 and DFDC datasets.

SFHA	DBP	TSCA	CDF2	DFDC
✗	✓	✓	0.7427	0.6779
✓	✗	✓	0.7711	0.6933
✓	✓	✗	0.7832	0.7146
✓	✓	✓	0.8111	0.7307

Table 4. Table 4: Ablation study on TSCA: performance vs. computational cost.

Model	CDF2	DFDC	FLOPs	Parameters
CCA-Fusion	0.7949	0.7216	2.28G	22.19M
w/o TSCA	0.7832	0.7146	1.26G	6.46M
ours	0.8111	0.7307	1.27G	6.64M

Equations

$${\mathrm{M}}_{\mathrm{freq}}={\mathcal{K}}_f\left(\mathcal{U}\left(\mathrm{ReLU}\left({\tilde{\mathcal{K}}}_f\left(\mathrm{Haar}({\mathrm{X}}_{\text{low}})\right)\right)\right)\right),$$

$${\mathrm{M}}_{\text{spa}}={\mathcal{K}}_s\cdot (\mathrm{CoordAdd}([\frac{1}{C}\sum _{c=1}^C{\mathrm{X}}_{\text{low}},\mathrm{ma}{\mathrm{x}}_{\mathrm{c}=1}^{\mathrm{C}}{\mathrm{X}}_{\text{low}}])),$$

$${\mathrm{G}}_{\text{low}}=\sigma ({\mathcal{K}}_g\cdot [\alpha \cdot {\mathrm{M}}_{\mathrm{freq}},\beta \cdot {\mathrm{M}}_{\text{spa}}]),$$

$${\mathrm{Y}}_{\text{low}}={\mathrm{X}}_{\text{low}}+{\mathrm{G}}_{\text{low}}\odot {\mathrm{X}}_{\text{low}},$$

$${\mathrm{D}}_{\text{main}}=\mathrm{ReLU}(\mathrm{BN}({\mathcal{K}}_m(\mathcal{B}({\mathrm{Y}}_{\text{low}})))),$$

$${\mathrm{D}}_{\mathrm{res}}=\mathrm{BN}({\mathcal{K}}_r(\mathcal{B}({\mathrm{Y}}_{\text{low}}))),$$

$${\mathrm{D}}_{\text{low}}=\mathrm{ReLU}({\mathrm{D}}_{\mathrm{main}}+{\mathrm{D}}_{\mathrm{res}}).$$

$${\mathrm{F}}_{\mathrm{proj}}=\mathcal{P}({\mathrm{Y}}_{\text{high}}),$$

$${\mathrm{F}}_{\mathrm{fuse}}=\mathcal{F}({\mathrm{D}}_{\text{low}},{\mathrm{Y}}_{\text{mid}}),$$

$$\{\begin{array}{rl}{\mathrm{Q}}_1& ={\mathcal{K}}_q({\mathrm{F}}_{\mathrm{fuse}}),\\ {\mathrm{K}}_2& ={\mathcal{K}}_k({\mathrm{F}}_{\mathrm{proj}}),\\ {\mathrm{V}}_2& ={\mathcal{K}}_v({\mathrm{F}}_{\mathrm{proj}}),\end{array}$$

$${\tilde{\mathrm{Q}}}_1,{\tilde{\mathrm{K}}}_2,{\tilde{\mathrm{V}}}_2=\mathcal{R}(\frac{1}{C}\sum _{c=1}^C({\mathrm{Q}}_1,{\mathrm{K}}_2,{\mathrm{V}}_2)),$$

$$\mathrm{Z}=(\mathrm{Softmax}(\frac{{\tilde{\mathrm{Q}}}_1\cdot {\tilde{\mathrm{K}}}_2^T}{\sqrt{d}})\cdot {\tilde{\mathrm{V}}}_2,$$

$${\mathrm{F}}_{\text{out}}=\alpha \cdot \mathcal{K}(\mathrm{Upsample}(\mathcal{R}(\mathrm{Z})))+(1-\alpha )\cdot \frac{{\mathrm{F}}_{\mathrm{fuse}}+{\mathrm{F}}_{\mathrm{proj}}}{2},$$

$${\mathcal{L}}_{\mathrm{gate}}=\frac{1}{3}\left(\ell ({\mathrm{G}}_{\mathrm{low}},y)+\ell ({\mathrm{G}}_{\mathrm{mid}},y)+\ell ({\mathrm{G}}_{\mathrm{high}},y)\right),$$

$$\ell (a,b)=\frac{1}{N}\sum _{i=1}^N(a_i-b_i{)}^2,$$

$${\mathcal{L}}_{\mathrm{cons}}=\frac{1}{3}\left(\ell ({\mathrm{G}}_{\mathrm{low}},{\mathrm{G}}_{\mathrm{mid}})+\ell ({\mathrm{G}}_{\mathrm{low}},{\mathrm{G}}_{\mathrm{high}})+\ell ({\mathrm{G}}_{\mathrm{mid}},{\mathrm{G}}_{\mathrm{high}})\right),$$

$${\mathcal{L}}_{\mathrm{total}}={\lambda }_{\mathrm{CE}}\cdot {\mathcal{L}}_{\mathrm{CE}}+{\lambda }_{\mathrm{gate}}\cdot {\mathcal{L}}_{\mathrm{gate}}+{\lambda }_{\mathrm{cons}}\cdot {\mathcal{L}}_{\mathrm{cons}},$$

Full text

1 Introduction

Recent advances in AI and deep learning have greatly improved deepfake technology, enabling highly realistic video and audio synthesis  [1, 2, 3]. While benefiting for film production and virtual entertainment, these developments raise privacy, ethical, and security concerns [4]. With better hardware and optimized algorithms, Real-Time Deepfakes [5] (RTDFs) now allow instant manipulation of live video and audio for broadcasts, video calls, and virtual reality. Researchers are working to overcome challenges like latency, synchronization, accuracy, and security through network optimization, hardware acceleration, and adaptive learning. As RTDFs expand applications in entertainment and remote communication, addressing risks and ensuring data security and ethical use remain crucial.

In recent years, deepfake detection has made significant progress, largely driven by the introduction of convolutional neural network-based approaches. Numerous studies have demonstrated that deep models can effectively identify subtle artifacts and manipulation traces introduced during forgery, achieving high detection accuracy and reasonable generalization across benchmark datasets such as FaceForensics++ [10] (FF++) and Celeb-DF [18]. However, most of these models rely on deep and complex architectures with high computational overhead, making them difficult to deploy in resource-constrained or latency-sensitive environments, and thereby limiting their applicability in real-time deepfake detection scenarios.

To support real-time applications, recent works have explored lightweight detection models by reducing input resolution, pruning backbones, or designing efficient architectures [6, 7]. While these approaches improve inference speed, they often suffer from limited representational capacity, reduced sensitivity to subtle artifacts, or loss of spatial and frequency details crucial for robust detection. Furthermore, many existing methods treat spatial and frequency inconsistencies independently, neglecting their joint contribution in revealing forgery clues. The key challenge, therefore, lies in designing real-time detection frameworks that maintain high accuracy and generalization while operating under constrained computational budgets. Addressing these issues, we propose a spatial-frequency hybrid aware network that explicitly models the interaction between spatial textures and frequency artifacts, enabling more accurate and efficient real-time deepfake detection.

Our method strikes an effective balance between performance and computational cost, outperforming most existing state-of-the-art approaches. We conducted extensive experiments on six widely used public benchmark datasets to comprehensively validate the effectiveness and efficiency of the proposed method. The main contributions of our work are summarized as follows:

[TABLE]

2 Related Work

2.1 Conventional Deepfake Detection

Deepfake detection has evolved significantly over time. Early approaches relied on low-level physical cues, such as unnatural facial movements [8] and compression artifacts [9]. More recent methods employ deep neural networks, particularly convolutional neural networks (CNNs) and transformer architectures, to capture complex spatial and temporal patterns. For instance, recent methods have introduced multi-head relative interaction mechanisms to analyze subtle noise patterns [11]. In addition, the combination of CNN-based feature extraction and the global information modeling capabilities of transformers [12] has significantly improved detection accuracy. With the emergence of diffusion models and large-scale generative models [23], the visual fidelity of forged content has surpassed traditional GAN-based methods. However, despite the remarkable performance of current detection models, their high computational complexity poses a challenge for real-time deployment, particularly in applications such as live content moderation and video conferencing.

2.2 Lightweight Detection Models

In recent years, researchers have focused on model compression, lightweight architectures, and attention mechanisms to improve detection efficiency without sacrificing accuracy. Early efforts centered on pruning, which simplifies conventional networks to reduce computational complexity. Efficient backbones, such as AlexNet [24], provided strong feature extraction while maintaining lower complexity. Attention mechanisms, exemplified by GoogLeNet [33], further enhanced the network’s ability to capture salient information with minimal parameter overhead, boosting performance in challenging detection scenarios. In deepfake detection, attention-based approaches have proven effective for extracting fine-grained features [13] and localizing manipulated regions [14]. Spatial and channel attention modules are particularly adept at emphasizing anomalies, while cross-layer attention facilitates multi-scale feature fusion. However, existing lightweight attention modules, such as CBAM [15], still incur significant computational costs, limiting their use in ultra-lightweight frameworks. Moreover, traditional fusion strategies often struggle to model complex feature interactions, further hindering detection performance.

3 Proposed method

The architecture of our proposed method is shown in Fig. 1. We employ a CNN backbone to extract multi-scale features and enhance artifacts using a spatial-frequency hybrid aware module. Low-level features are downsampled and fused with mid-level features, while high-level features are mapped into a compressed space. Selective token cross-attention efficiently models artifacts by linking fused mid- and low-level features with mapped high-level features. Finally, global pooling and an adaptive MLP classifier produce a binary classification result.

3.1 Spatial-Frequency Hybrid Aware Module

To highlight forged regions, we propose the Spatial-Frequency Hybrid Aware (SFHA) Module, which extracts wavelet-domain [16] features and spatial attention maps in parallel. These are fused into a dynamic gating map that emphasizes critical areas of the input features. Given a low-level feature map $\mathrm{X}_{\text{low}}\in\mathbb{R}^{C\times H\times W}$ from a CNN backbone, the wavelet branch first computes the frequency-aware attention map $\mathrm{M}_{\mathrm{freq}}$ as follows:

[TABLE]

where $\mathcal{K}_{f}$ denotes 2D convolution in the frequency domain, $\mathcal{U}$ denotes upsampling, $\tilde{\mathcal{K}}_{f}$ denotes 3D convolution, and $\mathrm{Haar}(\cdot)$ denotes channel-wise Haar wavelet transform, producing four sub-bands. The spatial branch then yields the spatial attention map $\mathrm{M}_{\mathrm{spa}}$ as follows:

[TABLE]

where $\mathcal{K}_{s}$ denotes 2D convolution in the spatial domain, and $\mathrm{CoordAdd}(\cdot)$ denotes the concatenation of normalized coordinate channels as extra positional cues. The notation $[,]$ represents channel-wise concatenation of two feature maps. The two attention maps are then adaptively fused to produce the low-level spatial–frequency gating $\mathrm{G}_{\text{low}}$ as follows:

[TABLE]

where $\mathcal{K}_{g}$ denotes 2D convolution, $\sigma$ denotes the sigmoid activation, and $\alpha,\beta$ are learnable weights for fusing frequency- and spatial-domain attention. The resulting hybrid gating $\mathrm{G}_{\text{low}}$ is then applied to the input feature $\mathrm{X}_{\text{low}}$, and the enhanced output $\mathrm{Y}_{\text{low}}$ is computed with a residual connection as follows:

[TABLE]

where $\odot$ denotes element-wise multiplication. To account for varying receptive fields and semantics at different feature levels, we use different convolution kernel sizes in the spatial branch. This enables multi-scale spatial feature extraction and yields the spatial–frequency enhanced features $\mathrm{Y}_{\text{low}}$, $\mathrm{Y}_{\text{mid}}$, and $\mathrm{Y}_{\text{high}}$.

3.2 Downsample with BlurPool Module

To reduce aliasing and better retain structure and edge details during downsampling, we introduce the Downsample with BlurPool (DBP) module. The input feature $\mathrm{Y}_{\text{low}}$ is first smoothed by blur pooling [17], then passed through convolution and ReLU to obtain the main semantic representation $\mathrm{D}_{\text{mian}}$ as follows:

[TABLE]

where $\mathcal{K}_{m}$ denotes convolution, and $\mathcal{B}$ denotes blur pooling. To reduce information loss, we introduce a residual path that applies the same blur and linear projection to the original input $\mathrm{Y}_{\text{low}}$ as follows:

[TABLE]

this path acts as an information compensation branch. Unlike the previous design, we apply a standard $1\times 1$ convolution $\mathcal{K}_{r}$ to the blurred features to capture texture details, improving training stability and retaining low-level information. Finally, we sum the main and residual paths and apply ReLU to produce the downsampled output as follows:

[TABLE]

For improved multi-level fusion, we apply the Projection Module to high-level features for mapping, and use the Fusion Module to integrate low- and mid-level features, as detailed below:

[TABLE]

[TABLE]

where $\mathcal{P}$ denotes the Projection Module, comprising convolutional, normalization, and activation layers. $\mathcal{F}$ denotes the Fusion Module, which concatenates feature maps followed by convolutional layers.

3.3 Token-Selective Cross Attention Module

We propose the Token-Selective Cross Attention (TSCA) module to enhance low-level details with high-level semantics while reducing computation. TSCA compresses spatial information, selects key tokens, and performs cross-attention to compute Query, Key, and Value representations as follows:

[TABLE]

where $\mathrm{Q}_{1},\mathrm{K}_{2},\mathrm{V}_{2}\in\mathbb{R}^{d\times H\times W}$. $\mathcal{K}_{q}$, $\mathcal{K}_{k}$, and $\mathcal{K}_{v}$ denote three independent convolutional layers that generate the query, key, and value feature maps, respectively. The parameter $d$ is the compressed channel dimension. To further reduce computation, the Token Selection mechanism applies spatial average pooling to $\mathrm{Q}_{1}$, $\mathrm{K}_{2}$, and $\mathrm{V}_{2}$, producing token representations $\tilde{\mathrm{Q}}_{1}$, $\tilde{\mathrm{K}}_{2}$, and $\tilde{\mathrm{V}}_{2}$ as follows:

[TABLE]

where $\tilde{\mathrm{K}}_{2}\in\mathbb{R}^{d\times T}$ and $\tilde{\mathrm{Q}}_{1},\tilde{\mathrm{V}}_{2}\in\mathbb{R}^{T\times d}$, $T=t\times t$ is the number of selected tokens, with $t$ the pooling size. The operator $\mathcal{R}$ denotes reshaping. The standard scaled dot-product attention is then computed as follows:

[TABLE]

where $\mathrm{Z}$ denotes the semantically enriched token representation, $d$ is the dimensionality of the query and key embeddings. It is reshaped to spatial size ($t\times t$), upsampled to the original feature resolution, and passed through a $1\times 1$ convolution to adjust channels. Finally, it is fused with the original low-level features to produce the output $\mathrm{F}_{\text{out}}$ as follows:

[TABLE]

where $\alpha$ is a learnable scalar weight used for residual balancing, and $\mathcal{K}$ represents the convolution operation.

3.4 Loss functions

We train the model with a weighted combination of cross-entropy loss, gated regularization loss, and consistency loss. Cross-entropy guides real or fake classification, the regularization term controls attention activation, and consistency loss stabilizes predictions under variations.

3.4.1 Classification Loss.

We use standard binary cross-entropy loss $\mathcal{L}_{\mathrm{CE}}$ to supervise the classification of real and fake samples. This loss encourages the model to produce high confidence for correct predictions.

3.4.2 Gated Regularization Loss.

To constrain the response patterns between attention maps at different scales and image labels, we propose the gated regularization loss as follows:

[TABLE]

where $\mathrm{G}_{\mathrm{low}},\mathrm{G}_{\mathrm{mid}},\mathrm{G}_{\mathrm{high}}$ represent the globally averaged pooled vectors of the low-, mid-, and high-level spatial-frequency hybrid gating, respectively. The term $y\in\{0,1\}^{N\times 1}$ denotes the labels for positive and negative samples, and $\ell(\cdot,\cdot)$ represents the mean squared error loss as follows:

[TABLE]

this loss encourages the gated attention maps of positive samples to exhibit higher response values, while the responses for negative samples tend to be lower.

3.4.3 Consistency Loss.

We further introduce a consistency loss to maintain the stability and coherence of attention responses across scales. It is defined as follows:

[TABLE]

this loss ensures that the model consistently identifies discriminative regions at different scales. This contributes to improved overall robustness.

3.4.4 Total Loss.

The final total loss function is defined as the weighted sum of the three components:

[TABLE]

where $\lambda_{\mathrm{CE}},\lambda_{\mathrm{gate}},\lambda_{\mathrm{cons}}$ are the weights that balance the classification loss, gated regularization loss, and consistency loss, respectively.

4 Experients

4.1 Experimental Settings

4.1.1 Datasets.

The FF++ [10] dataset is used for deepfake detection, containing real and manipulated face videos produced with four different forgery techniques, making it one of the most widely adopted benchmarks in this field. For evaluating the generalization performance of deepfake detection models, we also consider several well-established datasets: the DeepFake Detection [20] (DFD), the Celeb-DF v2 [18] (CDF2), the DeepFake Detection Challenge Preview [21] (DFDCP), the DeepFake Detection Challenge [19] (DFDC), and the UAD Fake Video [22] (UADFV). To maintain consistency with previous work, we strictly follow the image cropping and data preprocessing protocols defined by DeepfakeBench [37].

4.1.2 Implementation Details.

The experiments are conducted using the PyTorch Lightning framework for model training, validation, and testing, running entirely on NVIDIA V100 GPUs. To ensure stability and reproducibility, all configurations are kept fixed throughout the training process. The input image size is set to 256×256, with a batch size of 32. The Adam optimizer is used with a learning rate of 0.0002. The model is optimized on the training dataset with the loss weights set as $\lambda_{\mathrm{CE}}=1.0$, $\lambda_{\mathrm{gate}}=0.3$ and $\lambda_{\mathrm{cons}}=0.2$. Its performance is evaluated on the validation set after each epoch using the Area Under the Curve (AUC). Computational complexity and model size are assessed by reporting the number of Floating Point Operations (FLOPs) and parameters. After training, the model is tested on a separate test set to evaluate its generalization ability. The checkpoint achieving the highest AUC on the validation set is saved and used for final evaluation on the test set, ensuring the model’s robustness and practical applicability.

4.2 Evaluations

Given the real-time requirements of our task, we focus on detectors with low inference costs. Large-scale vision-language and diffusion-based detectors, such as ASAP [23], often exceed 100 GFLOPs and billions of parameters, making them impractical for latency-sensitive applications. In contrast, spatial- and frequency-based detectors like UCF [31] and X-ray [29] operate under 10 GFLOPs with fewer than 100 million parameters, offering competitive accuracy and high efficiency. We thus select recent detectors with similarly low computational footprints as baselines for a fair comparison in real-time scenarios.

To comprehensively evaluate the forgery detection performance of various models within a unified training domain, we conducted in-domain testing on the FaceForensics++ dataset, with results presented in Table 1. Our observations indicate that the proposed method achieves superior performance across most forgery subsets, attaining an average AUC of 0.9867, surpassing multiple state-of-the-art approaches. Notably, it achieves an AUC of 0.9748 on the complex FF-NT subset, demonstrating strong adaptability and robustness.

We used t-SNE to visualize the feature distributions learned by different models, as shown in Fig. 2. AlexNet and Xception show significant overlap between real and fake samples, indicating limited discriminative ability. F3Net performs better but still mixes different forgery types. In contrast, our method produces well-separated clusters, clearly distinguishing real and various fake sources, demonstrating superior feature representation.

Table 2 compares our method with state-of-the-art models on six datasets. Compared to UCF, our method achieves AUC improvements of 1.63% on FF++, 7.07% on DFD, 6.76% on CDF2, 10.09% on DFDCP, and 1.28% on DFDC. On UADFV, our AUC is 0.9420, slightly lower than UCF’s 0.9528. Overall, our average AUC reaches 0.8682, which is 4.99% higher than UCF (0.8267) and 6.35% higher than FFD (0.8164). Notably, compared to lightweight models like AlexNet, which achieves an average AUC of 0.7903, our method delivers a 7.79% improvement in average AUC, highlighting its superior performance in terms of accuracy. In terms of efficiency, our model uses only 1.27 GFLOPs and 6.64 million parameters, reducing computation by 87.9% compared to UCF and 78.9% compared to FFD, while maintaining superior detection performance.

To provide a more intuitive comparison, Fig. 3 illustrates the relationship between average AUC and FLOPs. The green dot represents our model (“ours”), which achieves the highest AUC (0.8682) with only 1.27 GFLOPs and 6.64M parameters. In contrast, other high-performing models such as Xception, FFD, and UCF, while offering better AUC, require significantly higher computational resources. Our model strikes a superior balance between performance and efficiency, delivering high detection accuracy while reducing computational cost.

4.3 Ablation Study

We conducted ablation experiments to assess the contribution of each module in our model, as shown in Table 3. Removing the SFHA module results in significant AUC drops on CDF2 and DFDC, highlighting the importance of spatial-frequency features. Replacing the DBP module with standard max-pooling also degrades performance, demonstrating the superiority of our downsampling approach. Similarly, substituting the TSCA module with simple channel concatenation and convolution reduces accuracy, confirming the effectiveness of token-selective attention. Overall, the complete model consistently outperforms all variants, validating the design choices.

To demonstrate the effectiveness of our DBP module, Fig. 4 presents an ablation comparison with average pooling. Ablating the DBP module and replacing it with standard average pooling results in blurred features and noisy edges, leading to a loss of important forgery-related details. In contrast, the DBP module preserves sharper structures and clearer boundaries, highlighting its superior capability in retaining crucial forgery cues during downsampling.

To assess the efficiency and effectiveness of the TSCA module, we conducted two ablation studies shown in Table 4. The first variant, CCA-Fusion, uses standard cross-attention and increases both FLOPs and parameters significantly. The second, w/o TSCA, removes TSCA and replaces it with basic channel fusion, reducing complexity but lowering accuracy. In contrast, TSCA introduces minimal overhead. FLOPs rise from 1.26G to 1.27G and parameters from 6.46M to 6.64M, while notably boosting performance, with AUC gains on CDF2 from 0.7832 to 0.8111 and on DFDC from 0.7146 to 0.7307. This confirms that TSCA is both lightweight and effective for feature enhancement.

5 Conclusion

In this work, we propose SFMFNet, a lightweight and efficient real-time deepfake detection framework. SFMFNet integrates wavelet features and coordinate attention into a dynamic gating mechanism, enhancing the detection of manipulated regions. It also introduces a token-selective cross-attention module to enable efficient cross-scale feature interaction. Additionally, we design a residual downsampling module based on blur pooling, which preserves structural and edge details while reducing information distortion. Extensive experiments demonstrate that SFMFNet achieves an excellent balance between accuracy, speed, and generalization, making it highly suitable for real-world deployment.