Measuring Inaccuracy in Travel Demand Forecasting: Methodological Considerations Regarding Ramp Up and Sampling

TL;DR

This paper defends the use of forecast data at decision time and first-year traffic for measuring demand forecast inaccuracy, arguing that empirical evidence and practical considerations do not support ramp-up adjustments, and highlights potential biases in large-sample studies.

Contribution

It challenges common objections to measuring forecast inaccuracy using decision-time forecasts and first-year data, providing methodological insights and bias considerations.

Findings

Forecasts at decision time are appropriate for assessing decision reliability.

Ramp-up effects are not necessary to include in large-N studies.

Large samples tend to underestimate true forecast inaccuracy due to bias.

Abstract

Project promoters, forecasters, and managers sometimes object to two things in measuring inaccuracy in travel demand forecasting: (1) using the forecast made at the time of making the decision to build as the basis for measuring inaccuracy and (2) using traffic during the first year of operations as the basis for measurement. This paper presents the case against both objections. First, if one is interested in learning whether decisions about building transport infrastructure are based on reliable information, then it is exactly the traffic forecasted at the time of making the decision to build that is of interest. Second, although ideally studies should take into account so-called demand "ramp up" over a period of years, the empirical evidence and practical considerations do not support this ideal requirement, at least not for large-N studies. Finally, the paper argues that large samples of inaccuracy in travel demand forecasts are likely to be conservatively biased, i.e., accuracy in travel demand forecasts estimated from such samples would likely be higher than accuracy in travel demand forecasts in the project population. This bias must be taken into account when interpreting the results from statistical analyses of inaccuracy in travel demand forecasting.