Transfer Learning and Locally Linear Regression for Locally Stationary Time Series

TL;DR

This paper advances locally linear regression for locally stationary time series, providing theoretical insights and proposing transfer learning methods that improve estimation accuracy across heterogeneous, time-varying domains, supported by simulations and empirical data.

Contribution

It develops uniform convergence results for locally linear estimators in locally stationary settings and introduces bias-corrected transfer learning techniques for heterogeneous time series.

Findings

Smaller error terms compared to Nadaraya-Watson estimator

Bias correction improves transfer learning performance

Empirical results validate theoretical predictions

Abstract

This paper investigates locally linear regression for locally stationary time series and develops theoretical results for locally linear smoothing and transfer learning. Existing analyses have focused on local constant estimators and given samples, leaving the principles of transferring knowledge from auxiliary sources across heterogeneous time-varying domains insufficiently established. We derive uniform convergence for multivariate locally linear estimators under strong mixing. The resulting error expansion decomposes stochastic variation, smoothing bias, and a term induced by local stationarity. This additional term, originating from the locally stationary structure, has smaller order than in the Nadaraya-Watson benchmark, explaining the improved local linear performance. Building on these results, we propose bias-corrected transfer learned estimators that connect a sparsely observed series with densely observed related sources through a smoothly varying bias function defined over rescaled time and covariates. An additional refinement shows how local temporal adjustment of this bias enhances stability and enables efficient information borrowing across domains. Simulation studies and an empirical analysis of international fuel prices support the theoretical predictions and demonstrate the practical advantages of transfer learning.