Model Selection in Time Series Analysis: Using Information Criteria as an Alternative to Hypothesis Testing

TL;DR

This paper evaluates the effectiveness of information criteria as an alternative to hypothesis testing for model selection in time series analysis, through Monte Carlo experiments and comparison of strategies.

Contribution

It introduces and assesses formal model selection techniques using information criteria and cross-validation in the context of time series data.

Findings

Information criteria outperform hypothesis testing in model selection accuracy.

Strategies based on information criteria better identify variable relations and unit root presence.

Monte Carlo experiments demonstrate the advantages of the proposed methods.

Abstract

The issue of model selection in applied research is of vital importance. Since the true model in such research is not known, which model should be used from among various potential ones is an empirical question. There might exist several competitive models. A typical approach to dealing with this is classic hypothesis testing using an arbitrarily chosen significance level based on the underlying assumption that a true null hypothesis exists. In this paper we investigate how successful this approach is in determining the correct model for different data generating processes using time series data. An alternative approach based on more formal model selection techniques using an information criterion or cross-validation is suggested and evaluated in the time series environment via Monte Carlo experiments. This paper also explores the effectiveness of deciding what type of general relation exists between two variables (e.g. relation in levels or relation in first differences) using various strategies based on hypothesis testing and on information criteria with the presence or absence of unit roots.