Meta-Neighborhoods

TL;DR

Meta-Neighborhoods is a semi-parametric approach that adaptively predicts based on local input neighborhoods, improving accuracy over traditional methods by leveraging learned induced neighborhoods and meta-learning.

Contribution

It introduces a novel semi-parametric method that generalizes k-nearest-neighbors with learned induced neighborhoods for more accurate local predictions.

Findings

Outperforms existing methods in predictive accuracy

Reduces memory and computation with induced neighborhoods

Effective meta-learning training mechanism

Abstract

Making an adaptive prediction based on one's input is an important ability for general artificial intelligence. In this work, we step forward in this direction and propose a semi-parametric method, Meta-Neighborhoods, where predictions are made adaptively to the neighborhood of the input. We show that Meta-Neighborhoods is a generalization of $k$-nearest-neighbors. Due to the simpler manifold structure around a local neighborhood, Meta-Neighborhoods represent the predictive distribution $p(y \mid x)$ more accurately. To reduce memory and computation overhead, we propose induced neighborhoods that summarize the training data into a much smaller dictionary. A meta-learning based training mechanism is then exploited to jointly learn the induced neighborhoods and the model. Extensive studies demonstrate the superiority of our method.