HybridVFL: Disentangled Feature Learning for Edge-Enabled Vertical Federated Multimodal Classification

TL;DR

HybridVFL is a new framework for vertical federated learning that improves multimodal classification accuracy on resource-limited devices by using feature disentanglement and a transformer-based fusion method.

Contribution

It introduces a novel client-server architecture with feature disentanglement and cross-modal transformers for enhanced privacy-preserving multimodal learning.

Findings

Significantly outperforms standard federated baselines on HAM10000 dataset.

Demonstrates the effectiveness of advanced fusion mechanisms in VFL.

Validates the importance of disentangled features for multimodal classification.

Abstract

Vertical Federated Learning (VFL) offers a privacy-preserving paradigm for Edge AI scenarios like mobile health diagnostics, where sensitive multimodal data reside on distributed, resource-constrained devices. Yet, standard VFL systems often suffer performance limitations due to simplistic feature fusion. This paper introduces HybridVFL, a novel framework designed to overcome this bottleneck by employing client-side feature disentanglement paired with a server-side cross-modal transformer for context-aware fusion. Through systematic evaluation on the multimodal HAM10000 skin lesion dataset, we demonstrate that HybridVFL significantly outperforms standard federated baselines, validating the criticality of advanced fusion mechanisms in robust, privacy-preserving systems.