Complexity of products: the effect of data regularisation

TL;DR

This paper investigates the impact of data regularisation techniques, specifically HMM regularisation, on economic complexity metrics and export network structures, and establishes the equivalence of SPSb with Nadaraya-Watson Kernel regression.

Contribution

It proves the convergence of SPSb to a simpler regression method, analyzes the effects of HMM regularisation on complexity metrics, and uncovers new effects of regularisation on network nestedness.

Findings

HMM regularisation reduces data noise.

Regularisation increases nestedness in export networks.

SPSb is equivalent to Nadaraya-Watson Kernel regression.

Abstract

Among several developments, the field of Economic Complexity (EC) has notably seen the introduction of two new techniques. One is the Bootstrapped Selective Predictability Scheme (SPSb), which can provide quantitative forecasts of the Gross Domestic Product of countries. The other, Hidden Markov Model (HMM) regularisation, denoises the datasets typically employed in the literature. We contribute to EC along three different directions. First, we prove the convergence of the SPSb algorithm to a well-known statistical learning technique known as Nadaraya-Watson Kernel regression. The latter has significantly lower time complexity, produces deterministic results, and it is interchangeable with SPSb for the purpose of making predictions. Second, we study the effects of HMM regularization on the Product Complexity and logPRODY metrics, for which a model of time evolution has been recently proposed. We find confirmation for the original interpretation of the logPRODY model as describing the change in the global market structure of products with new insights allowing a new interpretation of the Complexity measure, for which we propose a modification. Third, we explore new effects of regularisation on the data. We find that it reduces noise, and observe for the first time that it increases nestedness in the export network adjacency matrix.