VNIbCReg: VICReg with Neighboring-Invariance and better-Covariance Evaluated on Non-stationary Seismic Signal Time Series

TL;DR

This paper introduces VNIbCReg, a self-supervised learning method combining VICReg and TNC, tailored for non-stationary seismic time series, demonstrating improved encoding of non-stationary signals.

Contribution

It proposes a novel SSL approach that integrates VICReg with neighboring-invariance and better covariance handling, specifically designed for non-stationary time series.

Findings

Effective on non-stationary seismic signals

Outperforms traditional VICReg in non-stationary contexts

Combines strengths of VICReg and TNC for improved encoding

Abstract

One of the latest self-supervised learning (SSL) methods, VICReg, showed a great performance both in the linear evaluation and the fine-tuning evaluation. However, VICReg is proposed in computer vision and it learns by pulling representations of random crops of an image while maintaining the representation space by the variance and covariance loss. However, VICReg would be ineffective on non-stationary time series where different parts/crops of input should be differently encoded to consider the non-stationarity. Another recent SSL proposal, Temporal Neighborhood Coding (TNC) is effective for encoding non-stationary time series. This study shows that a combination of a VICReg-style method and TNC is very effective for SSL on non-stationary time series, where a non-stationary seismic signal time series is used as an evaluation dataset.