Average-case Complexity of Teaching Convex Polytopes via Halfspace Queries

TL;DR

This paper studies the average-case complexity of teaching convex polytopes via halfspace queries, revealing that it is proportional to the dimension, contrasting with the worst-case complexity which depends on the number of halfspaces.

Contribution

It introduces novel geometric techniques to analyze average-case teaching complexity, providing tight bounds and generalizing classical results in computational geometry.

Findings

Average-case teaching complexity is Θ(d).

Worst-case teaching complexity is Θ(n).

Average-case learning complexity depends on query type: Θ(n) for i.i.d., Θ(d log n) for active queries.

Abstract

We examine the task of locating a target region among those induced by intersections of $n$ halfspaces in $\mathbb{R}^d$. This generic task connects to fundamental machine learning problems, such as training a perceptron and learning a $\phi$-separable dichotomy. We investigate the average teaching complexity of the task, i.e., the minimal number of samples (halfspace queries) required by a teacher to help a version-space learner in locating a randomly selected target. As our main result, we show that the average-case teaching complexity is $\Theta(d)$, which is in sharp contrast to the worst-case teaching complexity of $\Theta(n)$. If instead, we consider the average-case learning complexity, the bounds have a dependency on $n$ as $\Theta(n)$ for \tt{i.i.d.} queries and $\Theta(d \log(n))$ for actively chosen queries by the learner. Our proof techniques are based on novel insights from computational geometry, which allow us to count the number of convex polytopes and faces in a Euclidean space depending on the arrangement of halfspaces. Our insights allow us to establish a tight bound on the average-case complexity for $\phi$-separable dichotomies, which generalizes the known $\mathcal{O}(d)$ bound on the average number of "extreme patterns" in the classical computational geometry literature (Cover, 1965).