A complexity analysis of statistical learning algorithms

TL;DR

This paper applies information-based complexity analysis to support vector machine algorithms, providing a comprehensive framework for understanding their computational complexity and efficiency in statistical learning tasks.

Contribution

It introduces a novel complexity analysis framework for SVMs and other machine learning algorithms, considering higher order operations and scaling to optimize error.

Findings

Complexity measures for SVMs are developed considering higher order operations.

Scaling complexities can minimize error in statistical learning algorithms.

Application demonstrated in biomedical informatics context.

Abstract

We apply information-based complexity analysis to support vector machine (SVM) algorithms, with the goal of a comprehensive continuous algorithmic analysis of such algorithms. This involves complexity measures in which some higher order operations (e.g., certain optimizations) are considered primitive for the purposes of measuring complexity. We consider classes of information operators and algorithms made up of scaled families, and investigate the utility of scaling the complexities to minimize error. We look at the division of statistical learning into information and algorithmic components, at the complexities of each, and at applications to support vector machine (SVM) and more general machine learning algorithms. We give applications to SVM algorithms graded into linear and higher order components, and give an example in biomedical informatics.