Example Selection For Dictionary Learning
Tomoki Tsuchida, Garrison W. Cottrell

TL;DR
This paper investigates active example selection strategies in dictionary learning with sparse activations, demonstrating that certain heuristics can accelerate the learning process compared to uniform sampling.
Contribution
It introduces and evaluates heuristic and saliency-inspired selection algorithms, showing their potential to speed up dictionary learning.
Findings
Some selection algorithms improve learning speed
Active selection can outperform uniform sampling
Speculation on why certain heuristics are effective
Abstract
In unsupervised learning, an unbiased uniform sampling strategy is typically used, in order that the learned features faithfully encode the statistical structure of the training data. In this work, we explore whether active example selection strategies - algorithms that select which examples to use, based on the current estimate of the features - can accelerate learning. Specifically, we investigate effects of heuristic and saliency-inspired selection algorithms on the dictionary learning task with sparse activations. We show that some selection algorithms do improve the speed of learning, and we speculate on why they might work.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications · Face and Expression Recognition · Machine Learning and Data Classification
MethodsSPEED: Separable Pyramidal Pooling EncodEr-Decoder for Real-Time Monocular Depth Estimation on Low-Resource Settings
