A dynamic programming approach for generalized nearly isotonic optimization

TL;DR

This paper introduces a generalized nearly isotonic optimization model and develops an efficient dynamic programming algorithm with optimal linear time complexity for solving large-scale shape constrained regression problems.

Contribution

It proposes a unified GNIO framework encompassing various shape constrained regressions and provides a novel, efficient dynamic programming solution with proven optimal O(n) complexity.

Findings

The algorithm achieves linear time complexity for special $\

$ ext{l}_2$-GNIO problems.

Numerical experiments demonstrate high efficiency and robustness compared to existing methods.

Abstract

Shape restricted statistical estimation problems have been extensively studied, with many important practical applications in signal processing, bioinformatics, and machine learning. In this paper, we propose and study a generalized nearly isotonic optimization (GNIO) model, which recovers, as special cases, many classic problems in shape constrained statistical regression, such as isotonic regression, nearly isotonic regression and unimodal regression problems. We develop an efficient and easy-to-implement dynamic programming algorithm for solving the proposed model whose recursion nature is carefully uncovered and exploited. For special $\ell_2$-GNIO problems, implementation details and the optimal ${\cal O}(n)$ running time analysis of our algorithm are discussed. Numerical experiments, including the comparisons among our approach, the powerful commercial solver Gurobi, and existing fast algorithms for solving $\ell_1$-GNIO and $\ell_2$-GNIO problems, on both simulated and real data sets, are presented to demonstrate the high efficiency and robustness of our proposed algorithm in solving large scale GNIO problems.