On the Role of Tikhonov Regularizations in Standard Optimization Problems

TL;DR

This paper systematically examines the role of Tikhonov regularization in optimization problems through examples involving condition number analysis, boundary layer phenomena in control problems, and optical device design, highlighting its importance and practical benefits.

Contribution

It provides a clear, systematic illustration of Tikhonov regularization's impact in optimization, including condition number prediction, boundary layer analysis, and real-world device design applications.

Findings

Tikhonov regularization improves condition numbers in symmetric matrices.

Regularization is crucial for solving boundary layer optimal control problems.

Guides the design of cost-effective gradient-index optical devices.

Abstract

Tikhonov regularization is a common technique used when solving poorly behaved optimization problems. Often, and with good reason, this technique is applied by practitioners in an ad hoc fashion. In this note, we systematically illustrate the role of Tikhonov regularizations in two simple, yet instructive examples. In one example, we use regular perturbation theory to predict the impact Tikhonov regularizations have on condition numbers of symmetric, positive semi-definite matrices. We then use a numerical example to confirm our result. In another example, we construct an exactly solvable optimal control problem that exhibits a boundary layer phenomenon. Since optimal control problems are rarely exactly solvable, this brings clarity to how vital Tikhonov regularizations are for the class of problems this example represents. We solve the problem numerically using MATLAB's built-in optimization software and compare results with a Galerkin-type/genetic algorithm. We find the second method generally outperforms MATLAB's optimization routines and we provide the MATLAB code used to generate the numerics. Finally, to demonstrate the utility of Tikhonov regularization from the second example in a more technologically realistic application, we show that regularization guides the design of gradient-index optical devices toward those which are significantly less expensive to fabricate.