Beyond S-curves: Recurrent Neural Networks for Technology Forecasting

TL;DR

This paper compares traditional S-curve models with modern machine learning approaches for technology forecasting, finding that autoencoders outperform S-curves, especially for emerging technologies, offering more accurate predictions.

Contribution

It introduces an autoencoder-based forecasting method that leverages recent time series advances and demonstrates its superior accuracy over S-curves and ARIMA models, particularly for emerging technologies.

Findings

Autoencoder reduces MAPE by 13.5% compared to second-best.

Autoencoder performs best on emerging technologies with 18% lower MAPE.

S-curves have similar accuracy to ARIMA but are less effective for new technologies.

Abstract

Because of the considerable heterogeneity and complexity of the technological landscape, building accurate models to forecast is a challenging endeavor. Due to their high prevalence in many complex systems, S-curves are a popular forecasting approach in previous work. However, their forecasting performance has not been directly compared to other technology forecasting approaches. Additionally, recent developments in time series forecasting that claim to improve forecasting accuracy are yet to be applied to technological development data. This work addresses both research gaps by comparing the forecasting performance of S-curves to a baseline and by developing an autencoder approach that employs recent advances in machine learning and time series forecasting. S-curves forecasts largely exhibit a mean average percentage error (MAPE) comparable to a simple ARIMA baseline. However, for a minority of emerging technologies, the MAPE increases by two magnitudes. Our autoencoder approach improves the MAPE by 13.5% on average over the second-best result. It forecasts established technologies with the same accuracy as the other approaches. However, it is especially strong at forecasting emerging technologies with a mean MAPE 18% lower than the next best result. Our results imply that a simple ARIMA model is preferable over the S-curve for technology forecasting. Practitioners looking for more accurate forecasts should opt for the presented autoencoder approach.