Senior Thesis for Haverford College Convex Optimization, Newton's Method and Interior Point Method

TL;DR

This thesis provides a comprehensive overview of convex optimization, including convex sets, functions, duality, and interior-point methods, with proofs and problem examples to illustrate key concepts.

Contribution

It offers a detailed, step-by-step explanation of convex optimization theory and the development of interior-point algorithms, including proofs and problem formulations.

Findings

Proved key theorems on convex sets and functions.

Demonstrated the existence and uniqueness of solutions in convex optimization.

Constructed interior-point method using barrier functions and central path.

Abstract

This paper consists of four general parts: convex sets; convex functions; convex optimization; and the interior-point algorithm. I will start by introducing the definition of convex sets and give three common convex set examples which will be used later in this paper, then prove the significant separating and supporting hyperplane theorems. Stepping into convex functions, in addition to offering definitions, I will also prove the first order and second-order conditions for convexity of a function, and then introduce a couple of examples that will be used in a convex optimization problem later. Next, I will provide the official definition of convex optimization problems and prove some characteristics they have, including the existence (through optimality criterion) and the uniqueness of a solution. I will also generate two convex optimization problems, one of which cannot be simply solved and requires additional skills. Afterward, I will introduce duality, for the sake of constructing the interior-point method. In the last section, I will first present the descent method and Newton's method, which serve as the foundation of the interior-point method. Then, I will show how to use the logarithmic barrier function and the central path to build up the interior-point method.