Sparse Trace Norm Regularization

TL;DR

This paper introduces a convex regularization approach combining trace norm and l1-norm for estimating multiple predictive functions with a sparse low-rank coefficient matrix, supported by theoretical analysis and efficient algorithms.

Contribution

It proposes a novel convex optimization framework for nonparametric regression that enforces sparsity and low-rank structure in the coefficient matrix, with proven performance bounds.

Findings

Algorithms based on AG and ADMM are efficient and effective.

Theoretical bounds demonstrate the method's performance under certain assumptions.

Simulation studies confirm the practical utility of the proposed approach.

Abstract

We study the problem of estimating multiple predictive functions from a dictionary of basis functions in the nonparametric regression setting. Our estimation scheme assumes that each predictive function can be estimated in the form of a linear combination of the basis functions. By assuming that the coefficient matrix admits a sparse low-rank structure, we formulate the function estimation problem as a convex program regularized by the trace norm and the $\ell_1$-norm simultaneously. We propose to solve the convex program using the accelerated gradient (AG) method and the alternating direction method of multipliers (ADMM) respectively; we also develop efficient algorithms to solve the key components in both AG and ADMM. In addition, we conduct theoretical analysis on the proposed function estimation scheme: we derive a key property of the optimal solution to the convex program; based on an assumption on the basis functions, we establish a performance bound of the proposed function estimation scheme (via the composite regularization). Simulation studies demonstrate the effectiveness and efficiency of the proposed algorithms.