Revisiting Logistic-softmax Likelihood in Bayesian Meta-Learning for Few-Shot Classification

TL;DR

This paper revisits and improves the logistic-softmax likelihood for Bayesian meta-learning in few-shot classification, enabling better uncertainty control and demonstrating superior performance on benchmarks.

Contribution

The authors redesign logistic-softmax likelihood to include a temperature parameter, establishing its relation to softmax and integrating it into Gaussian process meta-learning with data augmentation.

Findings

Achieves well-calibrated uncertainty estimates

Demonstrates comparable or superior results on benchmarks

Provides theoretical insights linking softmax and logistic-softmax

Abstract

Meta-learning has demonstrated promising results in few-shot classification (FSC) by learning to solve new problems using prior knowledge. Bayesian methods are effective at characterizing uncertainty in FSC, which is crucial in high-risk fields. In this context, the logistic-softmax likelihood is often employed as an alternative to the softmax likelihood in multi-class Gaussian process classification due to its conditional conjugacy property. However, the theoretical property of logistic-softmax is not clear and previous research indicated that the inherent uncertainty of logistic-softmax leads to suboptimal performance. To mitigate these issues, we revisit and redesign the logistic-softmax likelihood, which enables control of the \textit{a priori} confidence level through a temperature parameter. Furthermore, we theoretically and empirically show that softmax can be viewed as a special case of logistic-softmax and logistic-softmax induces a larger family of data distribution than softmax. Utilizing modified logistic-softmax, we integrate the data augmentation technique into the deep kernel based Gaussian process meta-learning framework, and derive an analytical mean-field approximation for task-specific updates. Our approach yields well-calibrated uncertainty estimates and achieves comparable or superior results on standard benchmark datasets. Code is publicly available at \url{https://github.com/keanson/revisit-logistic-softmax}.