Achievable Error Exponents for Two-Phase Multiple Classification

TL;DR

This paper introduces a two-phase sequential classification test that achieves optimal error exponents in $M$-ary classification, improving efficiency by eliminating the need for a reject option in the second phase.

Contribution

The paper proposes a novel two-phase test for $M$-ary classification that attains optimal error exponents without requiring a reject option in the second phase.

Findings

The two-phase test matches the performance of the optimal sequential test.

The test does not need a reject option in the second phase to achieve optimal error exponents.

Analytical and numerical results confirm the test's optimality and efficiency.

Abstract

We revisit $M$-ary classification of Gutman (TIT 1989), where one is tasked to determine whether a testing sequence is generated with the same distribution as one of the $M$ training sequences or not. Our main result is a two-phase test, its theoretical analysis and its optimality guarantee. Specifically, our two-phase test is a special case of a sequential test with only two decision time points: the first phase of our test is a fixed-length test with a reject option, the second-phase of our test proceeds only if a reject option is decided in the first phase and the second phase of our test does \emph{not} allow a reject option. To provide theoretical guarantee for our test, we derive achievable error exponents using the method of types and derive a converse result for the optimal sequential test using the techniques recently proposed by Hsu, Li and Wang (ITW, 2022) for binary classification. Analytically and numerically, we show that our two phase test achieves the performance of an optimal sequential test with proper choice of test parameters. In particular, similarly as the optimal sequential test, our test does not need a final reject option to achieve the optimal error exponent region while an optimal fixed-length test needs a reject option to achieve the same region. Finally, we specialize our results to binary classification when $M=2$ and to $M$-ary hypothesis testing when the ratio of the lengths of training sequences and testing sequences tends to infinity so that generating distributions can be estimated perfectly.