The Open-World Lottery Ticket Hypothesis for OOD Intent Classification

TL;DR

This paper extends the Lottery Ticket Hypothesis to open-world scenarios, showing that pruned, calibrated subnetworks improve out-of-domain intent classification by better distinguishing in- and out-of-domain data.

Contribution

It introduces the open-world Lottery Ticket Hypothesis and demonstrates that pruning overparameterized models yields calibrated subnetworks for OOD detection.

Findings

Calibrated subnetworks improve OOD detection accuracy.

Pruning overparameterized models enhances confidence calibration.

Consistent improvements over baseline methods on real-world datasets.

Abstract

Most existing methods of Out-of-Domain (OOD) intent classification rely on extensive auxiliary OOD corpora or specific training paradigms. However, they are underdeveloped in the underlying principle that the models should have differentiated confidence in In- and Out-of-domain intent. In this work, we shed light on the fundamental cause of model overconfidence on OOD and demonstrate that calibrated subnetworks can be uncovered by pruning the overparameterized model. Calibrated confidence provided by the subnetwork can better distinguish In- and Out-of-domain, which can be a benefit for almost all post hoc methods. In addition to bringing fundamental insights, we also extend the Lottery Ticket Hypothesis to open-world scenarios. We conduct extensive experiments on four real-world datasets to demonstrate our approach can establish consistent improvements compared with a suite of competitive baselines.