XTransfer: Modality-Agnostic Few-Shot Model Transfer for Human Sensing at the Edge

TL;DR

XTransfer introduces a novel, resource-efficient method for modality-agnostic, few-shot transfer of pre-trained models to human sensing tasks on edge devices, overcoming data scarcity and resource constraints.

Contribution

It presents the first approach enabling flexible, few-shot, modality-agnostic model transfer with a resource-efficient design for human sensing at the edge.

Findings

XTransfer achieves state-of-the-art performance across diverse datasets.

It significantly reduces sensor data collection and training costs.

The method enables effective model transfer with minimal sensor data.

Abstract

Deep learning for human sensing on edge systems presents significant potential for smart applications. However, its training and development are hindered by the limited availability of sensor data and resource constraints of edge systems. While transferring pre-trained models to different sensing applications is promising, existing methods often require extensive sensor data and computational resources, resulting in high costs and limited transferability. In this paper, we propose XTransfer, a first-of-its-kind method enabling modality-agnostic, few-shot model transfer with resource-efficient design. XTransfer flexibly uses pre-trained models and transfers knowledge across different modalities by (i) model repairing that safely mitigates modality shift by adapting pre-trained layers with only few sensor data, and (ii) layer recombining that efficiently searches and recombines layers of interest from source models in a layer-wise manner to restructure models. We benchmark various baselines across diverse human sensing datasets spanning different modalities. The results show that XTransfer achieves state-of-the-art performance while significantly reducing the costs of sensor data collection, model training, and edge deployment.