WildFit: Autonomous In-situ Model Adaptation for Resource-Constrained IoT Systems

TL;DR

WildFit is an autonomous, resource-efficient framework enabling IoT devices like wildlife cameras to adapt to environmental changes in-situ, maintaining high accuracy without relying on cloud connectivity or extensive retraining.

Contribution

This paper introduces WildFit, a novel in-situ adaptation method combining background-aware synthesis and drift-aware fine-tuning for resource-constrained IoT systems.

Findings

Background-aware synthesis improves accuracy by 7.3%.

Drift-aware fine-tuning reduces updates by 50% with 1.5% higher accuracy.

End-to-end system outperforms domain adaptation by 20-35%, using only 11.2 Wh over 37 days.

Abstract

Resource-constrained IoT devices increasingly rely on deep learning models, however, these models experience significant accuracy drops due to domain shifts when encountering variations in lighting, weather, and seasonal conditions. While cloud-based retraining can address this issue, many IoT deployments operate with limited connectivity and energy constraints, making traditional fine-tuning approaches impractical. We explore this challenge through the lens of wildlife ecology, where camera traps must maintain accurate species classification across changing seasons, weather, and habitats without reliable connectivity. We introduce WildFit, an autonomous in-situ adaptation framework that leverages the key insight that background scenes change more frequently than the visual characteristics of monitored species. WildFit combines background-aware synthesis to generate training samples on-device with drift-aware fine-tuning that triggers model updates only when necessary to conserve resources. Our background-aware synthesis surpasses efficient baselines by 7.3% and diffusion models by 3.0% while being orders of magnitude faster, our drift-aware fine-tuning achieves Pareto optimality with 50% fewer updates and 1.5% higher accuracy, and the end-to-end system outperforms domain adaptation approaches by 20-35% while consuming only 11.2 Wh over 37 days-enabling battery-powered deployment.