# Interpretable Few-Shot Learning via Linear Distillation

## TL;DR

This paper introduces Linear Distillation Learning, a simple and interpretable method that enhances linear neural networks for few-shot learning tasks, demonstrating improved performance on MNIST and Omniglot datasets.

## Contribution

It proposes a novel linear distillation approach that improves interpretability and performance of linear models in few-shot learning scenarios.

## Key findings

- Outperforms classical Logistic Regression on MNIST and Omniglot
- Provides a mathematically tractable and interpretable model
- Effective in few-shot learning settings

## Abstract

It is important to develop mathematically tractable models than can interpret knowledge extracted from the data and provide reasonable predictions. In this paper, we present a Linear Distillation Learning, a simple remedy to improve the performance of linear neural networks. Our approach is based on using a linear function for each class in a dataset, which is trained to simulate the output of a teacher linear network for each class separately. We tested our model on MNIST and Omniglot datasets in the Few-Shot learning manner. It showed better results than other interpretable models such as classical Logistic Regression.