Self-concordant analysis for logistic regression

TL;DR

This paper extends the theoretical analysis of regression to logistic loss using self-concordant functions, enabling simpler derivations of results for regularized logistic regression in binary classification.

Contribution

It introduces a novel application of self-concordant analysis to logistic regression, bridging the gap with least-squares regression results.

Findings

Extended theoretical results from square loss to logistic loss.

Derived new bounds for regularized logistic regression.

Simplified analysis framework for binary classification models.

Abstract

Most of the non-asymptotic theoretical work in regression is carried out for the square loss, where estimators can be obtained through closed-form expressions. In this paper, we use and extend tools from the convex optimization literature, namely self-concordant functions, to provide simple extensions of theoretical results for the square loss to the logistic loss. We apply the extension techniques to logistic regression with regularization by the $\ell_2$-norm and regularization by the $\ell_1$-norm, showing that new results for binary classification through logistic regression can be easily derived from corresponding results for least-squares regression.