# Scale-Adaptive Neural Dense Features: Learning via Hierarchical Context Aggregation

## TL;DR

This paper introduces SAND features, a hierarchical, deep learning-based feature extraction method that leverages sparse relative labels to adapt to various vision tasks, improving performance with minimal additional training.

## Contribution

The authors propose SAND features, a novel deep learning approach that uses hierarchical context aggregation and sparse relative labels for versatile feature extraction across multiple vision tasks.

## Key findings

- SAND features outperform baseline methods in several tasks.
- Incorporating SAND features improves disparity estimation and semantic segmentation.
- The approach requires little additional training for different applications.

## Abstract

How do computers and intelligent agents view the world around them? Feature extraction and representation constitutes one the basic building blocks towards answering this question. Traditionally, this has been done with carefully engineered hand-crafted techniques such as HOG, SIFT or ORB. However, there is no ``one size fits all'' approach that satisfies all requirements. In recent years, the rising popularity of deep learning has resulted in a myriad of end-to-end solutions to many computer vision problems. These approaches, while successful, tend to lack scalability and can't easily exploit information learned by other systems. Instead, we propose SAND features, a dedicated deep learning solution to feature extraction capable of providing hierarchical context information. This is achieved by employing sparse relative labels indicating relationships of similarity/dissimilarity between image locations. The nature of these labels results in an almost infinite set of dissimilar examples to choose from. We demonstrate how the selection of negative examples during training can be used to modify the feature space and vary it's properties. To demonstrate the generality of this approach, we apply the proposed features to a multitude of tasks, each requiring different properties. This includes disparity estimation, semantic segmentation, self-localisation and SLAM. In all cases, we show how incorporating SAND features results in better or comparable results to the baseline, whilst requiring little to no additional training. Code can be found at: https://github.com/jspenmar/SAND_features