Introducing the Partitioned Equivalence Test: Artificial Intelligence in Automatic Passenger Counting Validation

TL;DR

This paper introduces the partitioned equivalence test, an extension of the standard statistical test, designed to improve efficiency and reduce costs in validating automatic passenger counting systems, especially when using AI-assisted workflows.

Contribution

The paper presents a novel extension to the equivalence test that allows for arbitrary pre-classification, including AI-assisted methods, maintaining low user risk while reducing validation effort and costs.

Findings

Reduces validation effort in APC systems

Enables AI-assisted workflows without increasing user risk

Compatible with manual and automated classification methods

Abstract

Automatic passenger counting (APC) in public transport has been introduced in the 1970s and has been rapidly emerging in recent years. APC systems, like all other measurement devices, are susceptible to error, which is treated as random noise and is required to not exceed certain bounds. The demand for very low errors is especially fueld by applications like revenue sharing, which is in the billions, annually. As a result, both the requirements as well as the costs heavily increased. In this work, we address the latter problem and present a solution to increase the efficiency of initial or recurrent (e.g. yearly or more frequent) APC validation. Our new approach, the partitioned equivalence test, is an extension to this widely used statistic hypothesis test and guarantees the same bounded, low user risk while reducing effort. This can be used to either cut costs or to extend validation without cost increase. It involves a pre-classification step, which itsself can be arbitrary, so we evaluated several use cases: entirely manual and algorithmic, artificial intelligence assisted workflows. For former, by restructuring the evaluation of manual counts, our new statistical test can be used as a drop-in replacement for existing test procedures. The largest savings, however, result from latter algorithmic use cases: Due to the user risk being as bounded as in the original equivalence test, no additional requirements are introduced. Algorithms are allowed to be failable and thus, our test does not require the availability of general artificial intelligence. All in all, automatic passenger counting as well as the equivalence test itself can both benefit from our new extension.