Transfer Learning with Self-Supervised Vision Transformers for Snake Identification

TL;DR

This paper explores using self-supervised Vision Transformers, specifically DINOv2, for snake species identification from images, demonstrating promising results in a large dataset through embedding analysis and linear classification.

Contribution

The study applies DINOv2 vision transformer embeddings to snake identification, showcasing their effectiveness and providing insights through embedding analysis in a large-scale dataset.

Findings

Achieved a score of 39.69 in the competition

Demonstrated the potential of DINOv2 embeddings for species classification

Provided analysis of embedding structure and its relation to classification performance

Abstract

We present our approach for the SnakeCLEF 2024 competition to predict snake species from images. We explore and use Meta's DINOv2 vision transformer model for feature extraction to tackle species' high variability and visual similarity in a dataset of 182,261 images. We perform exploratory analysis on embeddings to understand their structure, and train a linear classifier on the embeddings to predict species. Despite achieving a score of 39.69, our results show promise for DINOv2 embeddings in snake identification. All code for this project is available at https://github.com/dsgt-kaggle-clef/snakeclef-2024.