Conditional Probability Tree Estimation Analysis and Algorithms

TL;DR

This paper introduces a new online algorithm for efficiently estimating conditional probabilities over large label sets by constructing a logarithmic depth tree, with theoretical guarantees and empirical validation on a dataset with over a million labels.

Contribution

It presents the first online algorithm that adaptively builds a logarithmic depth tree for label probability estimation, with proven regret bounds and practical effectiveness.

Findings

Algorithm constructs a logarithmic depth tree in practice.

The method achieves low regret bounds theoretically.

Empirical tests show success on datasets with over 10^6 labels.

Abstract

We consider the problem of estimating the conditional probability of a label in time O(log n), where n is the number of possible labels. We analyze a natural reduction of this problem to a set of binary regression problems organized in a tree structure, proving a regret bound that scales with the depth of the tree. Motivated by this analysis, we propose the first online algorithm which provably constructs a logarithmic depth tree on the set of labels to solve this problem. We test the algorithm empirically, showing that it works succesfully on a dataset with roughly 106 labels.