Self-Supervised Learning at the Edge: The Cost of Labeling

TL;DR

This paper investigates the use of self-supervised learning techniques on edge devices, demonstrating that tailored SSL strategies can significantly reduce resource consumption while maintaining competitive performance.

Contribution

It provides an analysis of SSL methods' efficiency on resource-constrained edge devices and explores semi-supervised learning to lower energy costs in training.

Findings

SSL can reduce resource use by up to 4X on edge devices.

Tailored SSL strategies maintain competitive performance under limited resources.

Semi-supervised learning helps decrease overall energy consumption.

Abstract

Contrastive learning (CL) has recently emerged as an alternative to traditional supervised machine learning solutions by enabling rich representations from unstructured and unlabeled data. However, CL and, more broadly, self-supervised learning (SSL) methods often demand a large amount of data and computational resources, posing challenges for deployment on resource-constrained edge devices. In this work, we explore the feasibility and efficiency of SSL techniques for edge-based learning, focusing on trade-offs between model performance and energy efficiency. In particular, we analyze how different SSL techniques adapt to limited computational, data, and energy budgets, evaluating their effectiveness in learning robust representations under resource-constrained settings. Moreover, we also consider the energy costs involved in labeling data and assess how semi-supervised learning may assist in reducing the overall energy consumed to train CL models. Through extensive experiments, we demonstrate that tailored SSL strategies can achieve competitive performance while reducing resource consumption by up to 4X, underscoring their potential for energy-efficient learning at the edge.