A Data-Ensemble-Based Approach for Sample-Efficient LQ Control of Linear Time-Varying Systems

TL;DR

This paper introduces a novel, data-driven, and sample-efficient control framework for finite-horizon linear quadratic control of linear time-varying systems, leveraging duality and convex optimization to improve over existing methods.

Contribution

It develops a non-iterative SDP algorithm that computes optimal feedback gains directly from data without system identification, applicable to both LTV and LTI systems.

Findings

Achieves superior optimality and sample efficiency in simulations.

Provides global optimality guarantees for unknown LTV systems.

Improves over existing Q-learning methods for finite-horizon LTI control.

Abstract

This paper presents a sample-efficient, data-driven control framework for finite-horizon linear quadratic (LQ) control of linear time-varying (LTV) systems. In contrast to the time-invariant case, the time-varying LQ problem involves a differential Riccati equation (DRE) with time-dependent parameters and terminal boundary constraints. We formulate the LQ problem as a nonconvex optimization problem and conduct a rigorous analysis of its dual structure. By exploiting the inherent convexity of the dual problem and analyzing the KKT conditions, we derive an explicit relationship between the optimal dual solution and the parameters of the associated Q-function in time-varying case. This theoretical insight supports the development of a novel, sample-efficient, non-iterative semidefinite programming (SDP) algorithm that directly computes the optimal sequence of feedback gains from an ensemble of input-state data sequences without model identification. The resulting convex, data-dependent framework provides global optimality guarantees for completely unknown LTV systems. As a special case, the method also applies to finite-horizon LQ control of linear time-invariant (LTI) systems. In this setting, a single input-state trajectory suffices to identify the optimal LQ feedback policy, improving significantly over existing Q-learning approaches for finite horizon LTI systems that typically require data from multiple episodes. The approach provides a new optimization-based perspective on Q-learning in time-varying settings and contributes to the broader understanding of data-driven control in non-stationary environments. Simulation results show that, compared to recent methods, the proposed approach achieves superior optimality and sample efficiency on LTV systems, and indicates potential for stabilizing and optimal control of nonlinear systems.