Inexact Sequential Quadratic Optimization with Penalty Parameter Updates Within the QP Solve: Extended Version

TL;DR

This paper introduces an inexact SQP method with a dynamic penalty parameter update strategy, improving convergence and efficiency for large-scale nonlinear optimization problems using matrix-free subproblem solvers.

Contribution

It proposes a novel penalty parameter updating strategy within inexact SQP methods, enhancing convergence and robustness in large-scale nonlinear optimization.

Findings

The method achieves reliable convergence with inexact QP solves.

Dynamic penalty updates improve progress prediction toward optimality.

Numerical experiments demonstrate the effectiveness of the proposed approach.

Abstract

This paper focuses on the design of sequential quadratic optimization (commonly known as SQP) methods for solving large-scale nonlinear optimization problems. The most computationally demanding aspect of such an approach is the computation of the search direction during each iteration, for which we consider the use of matrix-free methods. In particular, we develop a method that requires an inexact solve of a single QP subproblem to establish the convergence of the overall SQP method. It is known that SQP methods can be plagued by poor behavior of the global convergence mechanism. To confront this issue, we propose the use of an exact penalty function with a dynamic penalty parameter updating strategy to be employed within the subproblem solver in such a way that the resulting search direction predicts progress toward both feasibility and optimality. We present our parameter updating strategy and prove that, under reasonable assumptions, the strategy does not modify the penalty parameter unnecessarily. We also discuss a matrix-free subproblem solver in which our updating strategy can be incorporated. We close the paper with a discussion of the results of numerical experiments that illustrate the benefits of our proposed techniques.

Figures

Tables

Table 1. Table 1: Performance comparison of SQuID and the proposed algorithm on feasible problems.

Problem type	Algorithm	Succeed	Fail	Infeasible	Total
Feasible hs problems	SQuID	110 (90.16%)	11 (9.02%)	1 (0.82%)	122
	Proposed	115 (91.20%)	11 (8.80%)	0	126

Table 2. Table 2: Performance comparison of SQuID and the proposed algorithm on infeasible problems.

Problem type	Algorithm	Succeed	Fail	Total
Infeasible hs problems (hs_inf)	SQuID	111 (90.24%)	12 (9.76%)	123
	Proposed	116 (92.10%)	10 (7.90%)	126

Table 3. Table 3: CUTEr 13 large scale problems.

Problem	# constraints	# variables	# equalities
DTOC1NA	3996	5998	3996
DTOC1NB	3996	5998	3996
DTOC1ND	3996	5998	3996
EG3	20000	10001	1
GILBERT	1	5000	1
JANNSON4	2	10000	0
LUKVLE1	9998	10000	9998
LUKVLE10	9998	10000	9998
LUKVLE3	2	10000	2
LUKVLE6	4999	9999	4999
LUKVLI13	6664	9998	0
LUKVLI3	2	10000	0
LUKVLI6	4999	9999	0

Table 4. Table 4: Test results on CUTEr 13 large scale problems.

Problem	# iter	# $f$	$f\left(x^{\ast }\right)$	$v\left(x^{\ast }\right)$	KKT	Final $\rho$
DTOC1NA	13	13	4.138866e+00	2.215482e$-$06	1.878763e$-$05	0.751447
DTOC1NB	13	13	7.138849e+00	4.835061e$-$07	3.550798e$-$05	0.849347
DTOC1ND	14	19	4.760303e+01	1.799990e$-$07	4.807062e$-$05	0.815373
EG3	10	10	8.048306e$-$06	0.000000e+00	7.317141e$-$05	0.479603
GILBERT	74	74	2.459468e+03	2.170219e$-$08	4.047202e$-$06	0.024360
JANNSON4	79	80	9.801970e+03	6.956902e$-$08	1.830188e$-$05	0.009923
LUKVLE1	13	25	4.821043e$-$14	3.087659e$-$08	5.364314e$-$05	0.960000
LUKVLE10	191	191	3.534934e+03	2.224607e$-$09	9.783636e$-$05	0.282103
LUKVLE3	41	49	2.758658e+01	9.747758e$-$14	4.949755e$-$05	0.318856
LUKVLE6	39	68	6.286441e+05	1.436051e$-$12	6.916637e$-$05	0.360397
LUKVLI13	65	76	1.321855e+02	3.212148e$-$09	7.052314e$-$05	0.293858
LUKVLI3	70	78	1.157754e+01	9.010570e$-$13	6.644757e$-$05	0.442002
LUKVLI6	43	63	6.286441e+05	1.390753e$-$11	6.766861e$-$05	0.195366

Table 5. Table 5: CUTEr hs test results, 115 successful cases out of 126 problems.

Problem	# iter	# $f$	$f\left(x^{\ast }\right)$	$v\left(x^{\ast }\right)$	KKT	Final $\rho$
hs1	24	34	4.215353e$-$17	0.000000e+00	2.983621e$-$09	1.000000
hs10	8	9	$-$1.000001e+00	1.551523e$-$06	3.360370e$-$06	1.000000
hs100	11	22	6.806301e+02	2.403271e$-$06	8.021402e$-$06	0.540664
hs100lnp	10	31	6.806301e+02	9.444675e$-$06	1.615661e$-$06	0.150095
hs100mod	7	23	6.786796e+02	3.188191e$-$09	4.598100e$-$09	0.572194
hs101	51	145	1.809765e+03	3.225274e$-$08	4.579758e$-$07	0.000132
hs102	43	107	9.118803e+02	1.106847e$-$07	1.286046e$-$06	0.000274
hs103	45	135	5.436642e+02	3.902120e$-$06	7.284432e$-$06	0.000567
hs104	13	47	4.200002e+00	1.768636e$-$06	4.698347e$-$05	0.900000
hs105	20	990	1.044612e+03	1.067283e$-$07	8.477262e$-$05	0.004175
hs107	22	26	5.054972e+03	8.384681e$-$06	1.727293e$-$05	0.000688
hs108	18	32	$-$6.749664e$-$01	1.208205e$-$07	5.516680e$-$05	0.900000
hs109	78	220	5.362069e+03	7.846857e$-$07	8.667739e$-$05	0.128672
hs11	8	9	$-$8.498465e+00	1.225127e$-$07	1.349376e$-$06	0.218726
hs110	3	5	$-$4.577848e+01	0.000000e+00	6.067373e$-$09	1.000000
hs111	21	33	$-$4.776110e+01	4.440919e$-$06	2.258500e$-$05	0.042014
hs111lnp	20	30	$-$4.776119e+01	9.859016e$-$06	2.462468e$-$05	0.041440
hs112	21	23	$-$4.776117e+01	7.792329e$-$06	2.725259e$-$06	0.044630
hs113	16	17	2.430625e+01	1.363357e$-$06	9.915806e$-$06	0.387420
hs117	14	21	3.235087e+01	7.313662e$-$06	2.220430e$-$05	0.011973
hs118	19	20	9.329922e+02	9.973521e$-$06	1.820177e$-$06	0.064666
hs119	20	21	2.448993e+02	8.555172e$-$06	1.909610e$-$06	0.146587
hs12	5	9	$-$3.000000e+01	3.991066e$-$07	2.934243e$-$07	1.000000
hs14	20	89	1.393453e+00	7.451143e$-$06	4.922505e$-$06	0.414328
hs15	10	11	3.603797e+02	4.207535e$-$08	1.368772e$-$05	0.002465
hs16	20	21	2.314403e+01	9.780175e$-$06	9.391250e$-$05	0.014781
hs17	9	11	1.000000e+00	0.000000e+00	9.445528e$-$07	0.018248
hs18	7	8	5.000000e+00	2.771074e$-$09	4.843401e$-$08	1.000000
hs19	33	47	$-$6.961824e+03	8.499568e$-$06	8.737095e$-$06	0.000508
hs2	7	9	4.941229e+00	0.000000e+00	4.601831e$-$07	1.000000
hs20	25	26	4.019818e+01	2.882271e$-$06	9.114286e$-$05	0.008031
hs21	2	3	$-$9.996000e+01	4.440892e$-$16	4.999500e$-$09	1.000000
hs21mod	8	9	$-$9.596000e+01	2.220446e$-$16	4.371328e$-$18	0.162885
hs22	11	127	1.000002e+00	0.000000e+00	2.386921e$-$06	1.000000
hs23	20	21	1.999981e+00	9.518632e$-$06	5.376723e$-$06	0.282430
hs24	13	14	$-$9.998651e$-$01	2.585931e$-$12	7.171626e$-$05	0.531441
hs25	1	2	3.283500e+01	0.000000e+00	2.005805e$-$08	1.000000
hs26	13	28	2.172765e$-$10	4.361497e$-$06	2.031413e$-$07	1.000000
hs268	3	4	8.608487e$-$06	0.000000e+00	2.526260e$-$05	1.000000
hs27	7	11	4.000000e$-$02	1.781389e$-$19	5.580537e$-$05	1.000000
hs28	2	3	1.117108e$-$13	0.000000e+00	2.034355e$-$07	1.000000
hs29	8	9	$-$2.262742e+01	5.095551e$-$10	1.211382e$-$05	0.680483
hs3	7	8	2.338799e$-$04	0.000000e+00	9.672226e$-$05	1.000000
hs30	18	75	1.000103e+00	0.000000e+00	6.717072e$-$05	0.590490
hs31	9	11	6.000000e+00	1.192496e$-$08	7.648903e$-$05	0.104597
hs32	12	13	1.000880e+00	1.018075e$-$13	9.627754e$-$05	0.109419
hs33	9	93	$-$4.000000e+00	0.000000e+00	3.969005e$-$10	0.088629
hs34	19	28	$-$8.340328e$-$01	8.685515e$-$06	3.003686e$-$07	0.810000
hs35	1	2	1.111111e$-$01	0.000000e+00	8.332954e$-$05	1.000000
hs35i	1	2	1.111111e$-$01	0.000000e+00	8.332954e$-$05	1.000000
hs35mod	2	3	2.500000e$-$01	1.110223e$-$16	3.332426e$-$05	1.000000
hs36	16	17	$-$3.299993e+03	0.000000e+00	9.244868e$-$05	0.003757
hs37	8	11	$-$3.456000e+03	2.426503e$-$12	3.681286e$-$05	0.006363
hs38	37	57	4.607342e$-$11	0.000000e+00	4.104272e$-$06	1.000000
hs39	22	23	$-$1.000009e+00	9.375743e$-$06	6.443443e$-$06	0.687275
hs3mod	2	3	8.026142e$-$14	0.000000e+00	1.162326e$-$07	1.000000
hs4	2	3	2.666667e+00	0.000000e+00	3.149394e$-$15	0.228768
hs40	19	20	$-$2.500008e$-$01	9.306140e$-$06	2.419174e$-$06	1.000000
hs41	12	85	1.925925e+00	8.483081e$-$06	1.021823e$-$06	0.440257
hs42	4	18	1.385786e+01	2.177929e$-$10	1.588578e$-$05	0.313419
hs43	15	17	$-$4.399990e+01	1.315519e$-$08	3.235023e$-$05	0.324783
hs44	18	19	$-$1.499991e+01	5.129125e$-$06	1.160088e$-$05	0.079766
hs44new	20	21	$-$1.500002e+01	6.312695e$-$06	3.358322e$-$06	0.047101
hs45	17	18	1.000018e+00	6.508614e$-$06	1.250044e$-$05	0.590490
hs46	17	18	4.352054e$-$09	9.197854e$-$06	3.362897e$-$07	1.000000
hs47	16	20	1.134167e$-$09	9.982054e$-$06	1.208686e$-$06	0.135085
hs48	8	9	2.516051e$-$19	5.230553e$-$06	1.162349e$-$09	1.000000
hs49	13	14	2.791394e$-$07	7.660539e$-$12	3.325774e$-$05	1.000000
hs5	4	7	$-$1.913223e+00	0.000000e+00	3.354020e$-$05	0.656100
hs50	10	11	3.510103e$-$17	6.655753e$-$06	1.845457e$-$08	0.088629
hs51	2	3	6.496671e$-$17	1.204511e$-$08	9.999249e$-$09	1.000000
hs52	22	23	5.326608e+00	6.900875e$-$06	3.165318e$-$06	0.101755
hs53	21	22	4.092979e+00	9.854539e$-$06	4.525700e$-$06	0.129100
hs54	6	7	$-$1.561253e$-$01	4.934009e$-$10	9.544783e$-$05	1.000000
hs55	20	21	6.666664e+00	7.546464e$-$06	1.148850e$-$06	0.919332
hs56	9	11	$-$3.456002e+00	2.128966e$-$06	1.199327e$-$05	0.479441
hs57	1	2	3.064627e$-$02	0.000000e+00	2.696159e$-$06	1.000000
hs59	8	12	$-$7.802789e+00	0.000000e+00	3.334643e$-$06	0.900000
hs6	9	24	8.091820e$-$10	1.605262e$-$07	2.842575e$-$05	1.000000
hs60	5	6	3.256820e$-$02	9.958889e$-$08	1.894968e$-$07	1.000000
hs61	13	47	$-$1.436461e+02	1.588448e$-$06	4.775740e$-$07	0.338698
hs62	5	7	$-$2.627251e+04	1.526557e$-$16	4.647296e$-$07	0.001456
hs63	14	18	9.617152e+02	9.119443e$-$06	5.246424e$-$06	0.470499
hs64	43	44	6.299843e+03	1.254110e$-$08	7.139929e$-$05	0.041838
hs65	6	7	9.535284e$-$01	5.315868e$-$06	4.367160e$-$07	1.000000
hs66	6	10	5.181619e$-$01	7.088741e$-$06	1.275973e$-$06	0.900000
hs67	16	17	$-$1.162119e+03	0.000000e+00	6.495889e$-$05	1.000000
hs7	7	8	$-$1.732051e+00	9.245062e$-$07	1.247472e$-$06	1.000000
hs70	5	6	1.877865e$-$01	0.000000e+00	5.938470e$-$05	0.656100
hs71	20	29	1.701402e+01	6.474528e$-$06	6.121931e$-$07	0.585588
hs72	55	56	7.276756e+02	8.987068e$-$08	1.880063e$-$05	0.000015
hs73	18	19	2.989474e+01	1.903667e$-$09	8.426837e$-$06	0.032691
hs74	21	22	5.126498e+03	7.848646e$-$06	2.976747e$-$06	0.122491
hs75	181	531	5.174413e+03	5.201418e$-$06	7.131113e$-$07	0.000250
hs76	8	476	$-$4.681819e+00	5.280851e$-$07	9.252311e$-$05	0.387420
hs76i	7	80	$-$4.681711e+00	2.944135e$-$16	5.144605e$-$05	0.478297
hs77	19	21	2.415058e$-$01	7.872245e$-$06	6.733883e$-$07	1.000000
hs78	20	21	$-$2.919704e+00	8.018328e$-$06	2.979643e$-$06	0.656100
hs79	17	66	7.877677e$-$02	7.944258e$-$06	5.286831e$-$08	1.000000
hs8	15	48	$-$1.000000e+00	5.982302e$-$06	2.948125e$-$10	1.000000
hs80	17	84	5.394955e$-$02	7.682287e$-$06	2.102159e$-$07	1.000000
hs81	19	20	5.394951e$-$02	8.821814e$-$06	2.438733e$-$07	0.900000
hs86	19	375	$-$3.234871e+01	3.029890e$-$06	1.877376e$-$06	0.052335
hs88	33	37	1.362657e+00	2.312826e$-$12	3.800417e$-$07	0.000645
hs89	31	66	1.362657e+00	6.336861e$-$13	1.383187e$-$07	0.000693
hs9	2	3	$-$5.000000e$-$01	6.821210e$-$13	6.842740e$-$05	1.000000
hs90	33	49	1.362657e+00	7.690347e$-$12	4.771747e$-$06	0.000652
hs91	34	48	1.362657e+00	8.400233e$-$14	3.786938e$-$08	0.000660
hs92	33	44	1.362657e+00	1.792880e$-$11	1.663324e$-$05	0.000677
hs93	57	59	1.350759e+00	2.145353e$-$10	9.788245e$-$05	0.011972
hs95	26	27	1.567252e$-$02	3.024953e$-$10	3.415001e$-$05	0.011169
hs96	22	40	1.587067e$-$02	8.966170e$-$11	6.374995e$-$05	0.011070
hs97	34	60	4.071230e+00	5.619911e$-$08	7.635879e$-$05	0.001126
hs98	49	198	4.071243e+00	1.332748e$-$08	9.266559e$-$05	0.001105
hs99	18	19	$-$8.310799e+08	7.331983e$-$06	1.215967e$-$06	1.000000

Table 6. Table 6: CUTEr hs test results, 113 successful cases out of 126 problems with β ϕ = 0.5 subscript 𝛽 italic-ϕ 0.5 \beta_{\phi}=0.5 .

Problem	# iter	# $f$	$f\left(x^{\ast }\right)$	$v\left(x^{\ast }\right)$	KKT	Final $\rho$
hs1	24	34	4.215353e$-$17	0.000000e+00	2.983621e$-$09	1.000000
hs10	8	9	$-$1.000001e+00	1.551523e$-$06	3.360370e$-$06	1.000000
hs100	12	82	6.806301e+02	1.266571e$-$10	2.188096e$-$05	0.568586
hs100lnp	12	54	6.806301e+02	4.923703e$-$06	1.323439e$-$05	0.135085
hs100mod	7	23	6.786796e+02	1.128043e$-$09	2.302428e$-$09	0.665922
hs101	68	231	1.809744e+03	4.591308e$-$06	2.266775e$-$05	0.000153
hs104	13	21	4.200000e+00	2.658767e$-$10	7.616564e$-$08	0.718165
hs105	54	1060	1.044612e+03	2.560362e$-$08	3.225074e$-$05	0.003757
hs107	27	31	5.054978e+03	7.030538e$-$06	3.275745e$-$05	0.000761
hs108	18	38	$-$6.746727e$-$01	0.000000e+00	8.059244e$-$05	0.656100
hs109	91	208	5.362069e+03	5.071941e$-$06	7.901648e$-$05	0.143136
hs11	10	11	$-$8.498464e+00	1.123963e$-$08	1.512764e$-$07	0.251157
hs110	3	5	$-$4.577848e+01	0.000000e+00	6.067373e$-$09	1.000000
hs111	26	42	$-$4.776110e+01	2.671024e$-$06	1.824005e$-$06	0.044867
hs111lnp	29	45	$-$4.776109e+01	3.505706e$-$06	1.208402e$-$05	0.037393
hs112	24	27	$-$4.776118e+01	9.084188e$-$06	3.331079e$-$06	0.045371
hs113	15	16	2.430637e+01	3.236107e$-$06	3.610382e$-$05	0.387420
hs117	19	21	3.235008e+01	3.046132e$-$06	1.097142e$-$05	0.007070
hs118	24	25	9.329922e+02	7.148723e$-$06	1.631322e$-$06	0.076232
hs119	26	27	2.448993e+02	7.988931e$-$06	2.257583e$-$06	0.160258
hs12	5	9	$-$3.000000e+01	3.991066e$-$07	2.934243e$-$07	1.000000
hs14	23	119	1.393453e+00	7.451723e$-$06	5.102185e$-$06	0.429421
hs15	11	13	3.603798e+02	2.109424e$-$13	1.423044e$-$08	0.004071
hs16	24	25	2.314415e+01	7.882727e$-$06	7.697258e$-$05	0.014781
hs17	9	11	1.000000e+00	0.000000e+00	1.061723e$-$06	0.018248
hs18	9	10	5.000000e+00	1.234419e$-$08	1.001887e$-$07	1.000000
hs19	48	54	$-$6.961818e+03	3.596275e$-$06	2.625654e$-$06	0.000594
hs2	7	9	4.941229e+00	0.000000e+00	4.601831e$-$07	1.000000
hs20	32	33	4.019833e+01	2.088097e$-$06	9.533308e$-$05	0.009253
hs21	2	3	$-$9.996000e+01	4.440892e$-$16	4.999500e$-$09	1.000000
hs21mod	10	45	$-$9.596000e+01	0.000000e+00	8.731893e$-$11	0.189722
hs22	18	430	1.000005e+00	0.000000e+00	4.011241e$-$05	1.000000
hs23	16	17	2.000266e+00	0.000000e+00	6.251621e$-$05	0.150095
hs24	16	17	$-$9.998796e$-$01	0.000000e+00	8.782431e$-$05	0.729000
hs25	1	2	3.283500e+01	0.000000e+00	2.005805e$-$08	1.000000
hs26	13	28	2.172765e$-$10	4.361497e$-$06	2.031413e$-$07	1.000000
hs268	3	4	8.608487e$-$06	0.000000e+00	2.526260e$-$05	1.000000
hs27	7	11	4.000000e$-$02	1.781389e$-$19	5.580537e$-$05	1.000000
hs28	2	3	1.117108e$-$13	0.000000e+00	2.034355e$-$07	1.000000
hs29	8	9	$-$2.262742e+01	6.786021e$-$10	1.127105e$-$05	0.798638
hs3	7	8	2.338799e$-$04	0.000000e+00	9.672226e$-$05	1.000000
hs30	17	62	1.000015e+00	4.979195e$-$06	6.160904e$-$05	0.900000
hs31	8	10	5.999995e+00	7.977267e$-$07	1.615447e$-$05	0.122758
hs32	15	16	1.001279e+00	5.218048e$-$15	9.183154e$-$05	0.071790
hs33	4	5	$-$4.000003e+00	3.003592e$-$07	5.131525e$-$06	0.088629
hs34	23	28	$-$8.340328e$-$01	6.899262e$-$06	2.413293e$-$07	0.810000
hs35	1	2	1.111111e$-$01	0.000000e+00	8.332954e$-$05	1.000000
hs35i	1	2	1.111111e$-$01	0.000000e+00	8.332954e$-$05	1.000000
hs35mod	3	7	2.500000e$-$01	0.000000e+00	1.082126e$-$07	0.810000
hs36	19	20	$-$3.299992e+03	0.000000e+00	9.398680e$-$05	0.003757
hs37	8	18	$-$3.456000e+03	0.000000e+00	3.881216e$-$05	0.006363
hs38	38	58	7.867591e$-$14	0.000000e+00	1.866336e$-$07	1.000000
hs39	27	28	$-$1.000008e+00	7.670604e$-$06	5.813017e$-$06	0.757858
hs3mod	2	3	8.026142e$-$14	0.000000e+00	1.162326e$-$07	1.000000
hs4	2	3	2.666667e+00	0.000000e+00	3.149394e$-$15	0.228768
hs40	23	24	$-$2.500012e$-$01	7.931705e$-$06	2.342802e$-$06	1.000000
hs41	17	86	1.925925e+00	6.326517e$-$06	1.242304e$-$06	0.516082
hs42	4	18	1.385786e+01	2.198841e$-$10	3.639101e$-$06	0.367839
hs43	15	17	$-$4.399977e+01	5.769252e$-$10	7.534945e$-$05	0.325810
hs44	21	22	$-$1.500003e+01	9.486390e$-$06	1.097773e$-$06	0.007070
hs44new	21	22	$-$1.299990e+01	7.165140e$-$06	7.888032e$-$06	0.042391
hs45	18	19	1.000077e+00	0.000000e+00	2.427232e$-$05	0.531441
hs46	21	54	1.908729e$-$09	8.068528e$-$06	1.842347e$-$07	1.000000
hs47	20	28	1.495011e$-$10	6.225418e$-$06	3.890209e$-$07	0.109419
hs48	9	10	2.756164e$-$20	4.143635e$-$06	4.604039e$-$10	1.000000
hs49	17	143	2.791394e$-$07	4.239720e$-$12	3.325774e$-$05	1.000000
hs5	5	8	$-$1.913223e+00	0.000000e+00	3.368922e$-$08	0.590490
hs50	11	12	3.197115e$-$16	2.625468e$-$07	5.107095e$-$08	0.006363
hs51	2	3	5.165478e$-$17	4.610553e$-$09	9.999748e$-$09	1.000000
hs52	27	28	5.326601e+00	8.035451e$-$06	4.260170e$-$06	0.113117
hs53	26	27	4.092982e+00	9.247958e$-$06	4.663561e$-$06	0.144093
hs54	6	7	$-$1.561253e$-$01	9.712104e$-$11	9.544783e$-$05	1.000000
hs55	22	23	6.666665e+00	9.411365e$-$06	1.599698e$-$06	0.810000
hs56	9	11	$-$3.456007e+00	5.250197e$-$06	5.892236e$-$05	0.478297
hs57	1	2	3.064627e$-$02	0.000000e+00	2.696159e$-$06	1.000000
hs59	21	60	$-$7.802789e+00	5.169181e$-$08	4.257062e$-$06	1.000000
hs6	9	24	8.091820e$-$10	1.605262e$-$07	2.842575e$-$05	1.000000
hs60	5	6	3.256820e$-$02	9.958889e$-$08	1.894968e$-$07	1.000000
hs61	17	110	$-$1.436461e+02	5.755363e$-$06	2.279362e$-$06	0.446153
hs62	5	7	$-$2.627251e+04	1.526557e$-$16	4.647296e$-$07	0.001456
hs63	19	51	9.617152e+02	5.037713e$-$06	2.648938e$-$06	0.430467
hs64	45	46	6.299843e+03	2.414132e$-$08	8.508737e$-$05	0.048452
hs65	6	7	9.535284e$-$01	5.332807e$-$06	4.381076e$-$07	1.000000
hs66	6	44	5.181617e$-$01	2.335037e$-$06	1.396659e$-$06	0.900000
hs67	17	18	$-$1.162119e+03	6.817694e$-$06	6.292961e$-$06	1.000000
hs69	117	209	$-$9.567129e+02	4.448109e$-$13	9.574185e$-$05	0.000002
hs7	7	8	$-$1.732051e+00	9.245062e$-$07	1.247472e$-$06	1.000000
hs70	19	34	1.875462e$-$01	0.000000e+00	9.767005e$-$05	0.656100
hs71	26	36	1.701402e+01	7.582508e$-$06	6.233309e$-$07	0.645973
hs72	76	77	7.277010e+02	7.386719e$-$10	1.621663e$-$05	0.000018
hs73	18	21	2.989574e+01	1.459998e$-$10	3.044686e$-$05	0.030903
hs74	29	30	5.126498e+03	5.202308e$-$06	1.885142e$-$06	0.137573
hs76	17	325	$-$4.681787e+00	2.220446e$-$16	4.714559e$-$05	0.430467
hs76i	22	437	$-$4.681822e+00	2.293698e$-$06	8.936387e$-$05	0.387420
hs77	23	26	2.415057e$-$01	6.573382e$-$06	5.622722e$-$07	1.000000
hs78	24	25	$-$2.919703e+00	7.440837e$-$06	3.113752e$-$06	0.729000
hs79	19	31	7.877664e$-$02	6.433766e$-$06	1.264371e$-$07	1.000000
hs8	17	19	$-$1.000000e+00	8.877931e$-$06	4.375083e$-$10	1.000000
hs80	19	20	5.394949e$-$02	9.291850e$-$06	3.046230e$-$07	1.000000
hs81	23	24	5.394956e$-$02	7.427161e$-$06	2.011085e$-$07	0.900000
hs86	29	302	$-$3.234877e+01	7.952470e$-$06	3.592137e$-$06	0.038152
hs88	43	47	1.362657e+00	2.171298e$-$13	6.694802e$-$08	0.000701
hs89	37	72	1.362657e+00	5.467216e$-$13	4.170063e$-$07	0.000734
hs9	2	3	$-$5.000000e$-$01	6.821210e$-$13	6.842740e$-$05	1.000000
hs90	44	51	1.362657e+00	7.816595e$-$13	8.366618e$-$05	0.000699
hs91	47	59	1.362657e+00	1.651768e$-$14	1.537274e$-$08	0.000694
hs92	46	57	1.362657e+00	6.403740e$-$14	3.249272e$-$08	0.000696
hs93	59	61	1.350760e+02	4.464122e$-$10	8.911687e$-$05	0.011973
hs95	28	33	1.572551e$-$02	5.708108e$-$13	7.454286e$-$05	0.010023
hs96	28	36	1.570997e$-$02	8.056444e$-$12	2.606842e$-$05	0.011554
hs97	49	114	3.135805e+00	2.494120e$-$08	9.749914e$-$05	0.001008
hs98	78	359	3.135808e+00	7.729489e$-$09	2.828064e$-$05	0.000731
hs99	23	26	$-$8.310799e+08	8.955496e$-$06	1.485217e$-$06	1.000000

Table 7. Table 7: CUTEr hs test results, 111 successful cases out of 126 problems with β ϕ = 0.99 subscript 𝛽 italic-ϕ 0.99 \beta_{\phi}=0.99 .

Problem	# iter	# $f$	$f\left(x^{\ast }\right)$	$v\left(x^{\ast }\right)$	KKT	Final $\rho$
hs1	24	34	4.215353e$-$17	0.000000e+00	2.983621e$-$09	1.000000
hs10	8	9	$-$1.000001e+00	1.454144e$-$06	7.004923e$-$05	0.964609
hs100	7	16	6.806301e+02	3.072384e$-$06	3.762049e$-$06	0.410794
hs100lnp	8	28	6.806301e+02	1.243959e$-$06	1.726943e$-$06	0.150095
hs100mod	7	23	6.786796e+02	1.918734e$-$07	1.932158e$-$07	0.405803
hs104	9	14	4.200000e+00	7.857448e$-$11	1.338461e$-$09	0.852182
hs105	21	24	1.044612e+03	6.440900e$-$09	9.183609e$-$05	0.005154
hs107	8	11	5.054992e+03	4.009292e$-$06	2.372680e$-$05	0.000813
hs108	9	19	$-$8.660181e$-$01	8.033984e$-$06	3.876695e$-$05	1.000000
hs109	57	145	5.362069e+03	5.287754e$-$07	9.963619e$-$05	0.106742
hs11	7	8	$-$8.498465e+00	2.140915e$-$07	2.327186e$-$06	0.160679
hs110	3	5	$-$4.577848e+01	0.000000e+00	6.067373e$-$09	1.000000
hs111	13	23	$-$4.776109e+01	1.828854e$-$06	8.498378e$-$05	0.037562
hs111lnp	17	31	$-$4.776110e+01	9.677666e$-$07	9.685628e$-$05	0.036232
hs112	9	10	$-$4.776111e+01	6.414382e$-$06	1.509000e$-$06	0.033584
hs113	6	7	2.430621e+01	7.842192e$-$06	3.828328e$-$06	0.430467
hs117	5	10	3.234984e+01	2.854480e$-$07	9.921332e$-$05	0.016423
hs118	8	9	9.329922e+02	9.331469e$-$06	1.385033e$-$06	0.051875
hs119	11	12	2.448995e+02	5.010450e$-$06	8.440035e$-$07	0.105371
hs12	5	9	$-$3.000000e+01	3.972022e$-$07	3.691353e$-$05	0.947984
hs14	8	40	1.393453e+00	7.453297e$-$06	3.473740e$-$06	0.292307
hs15	20	21	3.064994e+02	3.356833e$-$07	7.885639e$-$05	0.000901
hs17	9	10	1.000000e+00	0.000000e+00	1.311337e$-$07	0.018248
hs18	6	7	5.000000e+00	0.000000e+00	4.941352e$-$08	1.000000
hs19	21	31	$-$6.961814e+03	2.065775e$-$07	1.789287e$-$05	0.000485
hs2	7	9	4.941229e+00	0.000000e+00	4.601831e$-$07	1.000000
hs20	16	17	4.019853e+01	1.032912e$-$06	9.289010e$-$05	0.006655
hs21	2	3	$-$9.996000e+01	0.000000e+00	1.059788e$-$07	1.000000
hs21mod	6	7	$-$9.596000e+01	2.220446e$-$16	1.217378e$-$18	0.137065
hs22	4	5	1.000000e+00	1.463081e$-$09	1.174890e$-$05	0.812471
hs23	9	10	1.999983e+00	8.315618e$-$06	9.354214e$-$06	0.245995
hs24	8	9	$-$9.999917e$-$01	1.786971e$-$11	2.342727e$-$05	1.000000
hs25	1	2	3.283500e+01	0.000000e+00	2.005805e$-$08	1.000000
hs26	13	28	2.172765e$-$10	4.361497e$-$06	2.031413e$-$07	1.000000
hs268	3	4	8.608487e$-$06	0.000000e+00	2.526260e$-$05	1.000000
hs27	7	11	4.000000e$-$02	1.781389e$-$19	5.580537e$-$05	1.000000
hs28	2	3	1.117108e$-$13	0.000000e+00	2.034355e$-$07	1.000000
hs29	8	9	$-$2.262742e+01	1.387226e$-$09	1.643885e$-$05	0.509159
hs3	7	8	2.338799e$-$04	0.000000e+00	9.672226e$-$05	1.000000
hs30	9	11	1.000002e+00	5.926657e$-$06	1.713723e$-$05	0.900000
hs31	6	8	6.000002e+00	0.000000e+00	3.527147e$-$06	0.078262
hs32	3	4	1.000217e+00	8.459899e$-$14	4.972783e$-$05	0.228768
hs33	4	6	$-$3.999996e+00	2.597922e$-$10	3.209629e$-$06	0.088629
hs34	8	15	$-$8.340328e$-$01	8.697522e$-$06	3.382240e$-$07	0.900000
hs35	1	2	1.111111e$-$01	0.000000e+00	8.332954e$-$05	1.000000
hs35i	1	2	1.111111e$-$01	0.000000e+00	8.332954e$-$05	1.000000
hs35mod	2	3	2.500037e$-$01	1.110223e$-$16	6.582735e$-$06	0.900000
hs36	9	10	$-$3.300001e+03	9.869097e$-$06	9.110111e$-$06	0.005726
hs37	7	8	$-$3.456000e+03	1.595168e$-$11	9.119249e$-$05	0.006363
hs38	37	58	6.716615e$-$09	0.000000e+00	9.209631e$-$05	1.000000
hs39	10	11	$-$1.000010e+00	9.589344e$-$06	5.495945e$-$06	0.573155
hs3mod	2	3	8.026142e$-$14	0.000000e+00	1.162326e$-$07	1.000000
hs4	2	3	2.666667e+00	0.000000e+00	3.149394e$-$15	0.228768
hs40	9	10	$-$2.500017e$-$01	7.450609e$-$06	2.239323e$-$06	1.000000
hs41	6	7	1.925926e+00	1.833479e$-$06	6.510768e$-$05	0.327095
hs42	4	18	1.385786e+01	2.149501e$-$10	4.274437e$-$05	0.234510
hs43	10	12	$-$4.399999e+01	1.159501e$-$10	2.722291e$-$06	0.313363
hs44	12	13	$-$1.500007e+01	9.977008e$-$06	9.552142e$-$06	0.109419
hs44new	13	14	$-$1.500005e+01	6.267152e$-$06	6.000245e$-$06	0.109419
hs45	14	15	1.000002e+00	0.000000e+00	6.280274e$-$07	0.506331
hs46	15	18	1.940719e$-$08	6.448602e$-$06	1.154214e$-$06	1.000000
hs47	15	20	4.310114e$-$09	2.798472e$-$06	6.038699e$-$06	0.313811
hs48	4	5	8.943420e$-$21	5.673768e$-$06	7.563741e$-$10	1.000000
hs49	13	14	2.791394e$-$07	7.657430e$-$12	3.325774e$-$05	1.000000
hs5	5	8	$-$1.913223e+00	0.000000e+00	4.292412e$-$06	0.387420
hs50	8	9	2.141999e$-$11	4.935936e$-$06	5.865571e$-$06	0.150095
hs51	2	3	6.496671e$-$17	1.204511e$-$08	9.999249e$-$09	1.000000
hs52	10	11	5.326603e+00	7.782006e$-$06	2.944367e$-$06	0.082081
hs53	9	10	4.092983e+00	9.355034e$-$06	2.718295e$-$06	0.086098
hs54	6	7	$-$1.561253e$-$01	8.805858e$-$10	9.544783e$-$05	1.000000
hs55	8	9	6.666669e+00	6.324330e$-$06	1.964817e$-$06	0.702330
hs56	7	9	$-$3.456003e+00	2.522962e$-$06	2.063936e$-$05	0.383819
hs57	1	2	3.064627e$-$02	0.000000e+00	2.696159e$-$06	1.000000
hs59	12	28	$-$7.802789e+00	1.375042e$-$09	7.514656e$-$07	1.000000
hs6	9	24	8.091820e$-$10	1.605262e$-$07	2.842575e$-$05	1.000000
hs60	5	6	3.256820e$-$02	9.958889e$-$08	1.894968e$-$07	1.000000
hs61	15	250	$-$1.436461e+02	1.592513e$-$06	3.957012e$-$07	0.282333
hs62	5	7	$-$2.627251e+04	1.526557e$-$16	4.647296e$-$07	0.001456
hs63	8	11	9.617152e+02	7.274287e$-$06	1.803124e$-$05	0.380778
hs64	43	44	6.299843e+03	9.138578e$-$09	8.249539e$-$05	0.031639
hs65	5	6	9.535288e$-$01	3.728086e$-$07	1.739860e$-$07	1.000000
hs66	3	8	5.181609e$-$01	6.792149e$-$06	1.674698e$-$06	1.000000
hs67	16	17	$-$1.162119e+03	0.000000e+00	3.710131e$-$07	1.000000
hs7	7	8	$-$1.732051e+00	9.245062e$-$07	1.247472e$-$06	1.000000
hs70	18	23	1.875514e$-$01	0.000000e+00	9.833571e$-$05	0.656100
hs71	10	17	1.701402e+01	5.412577e$-$06	2.626199e$-$05	0.387655
hs72	37	38	7.276793e+02	1.050624e$-$09	1.520144e$-$05	0.000015
hs73	5	6	2.989515e+01	9.876709e$-$07	1.702878e$-$05	0.029923
hs74	14	15	5.126498e+03	1.689201e$-$06	5.878016e$-$05	0.093021
hs75	138	482	5.174413e+03	8.891761e$-$06	2.216519e$-$05	0.000283
hs76	3	4	$-$4.681787e+00	3.049008e$-$16	1.646295e$-$05	0.531441
hs76i	2	3	$-$4.681771e+00	5.551115e$-$16	2.528737e$-$05	0.531441
hs77	10	12	2.415047e$-$01	5.346488e$-$06	2.731655e$-$06	1.000000
hs78	8	9	$-$2.919696e+00	6.902499e$-$06	1.418866e$-$06	0.329716
hs79	8	10	7.877678e$-$02	4.464599e$-$06	3.741699e$-$08	1.000000
hs8	7	8	$-$1.000000e+00	6.707219e$-$06	3.305371e$-$10	1.000000
hs80	8	11	5.394964e$-$02	5.273635e$-$06	1.839952e$-$07	1.000000
hs81	8	9	5.394952e$-$02	8.630110e$-$06	2.148511e$-$07	0.900000
hs86	3	4	$-$3.234849e+01	4.202104e$-$06	1.382075e$-$05	0.071790
hs88	25	39	1.362657e+00	2.457157e$-$14	2.541669e$-$08	0.000656
hs89	25	60	1.362657e+00	1.128935e$-$13	5.610106e$-$08	0.000678
hs9	2	3	$-$5.000000e$-$01	6.821210e$-$13	6.842740e$-$05	1.000000
hs90	25	43	1.362657e+00	2.226114e$-$14	2.424408e$-$08	0.000655
hs91	26	51	1.362657e+00	2.715207e$-$11	6.579864e$-$06	0.000603
hs92	25	36	1.362657e+00	2.460974e$-$14	2.543059e$-$08	0.000656
hs93	64	66	1.350760e+02	4.470868e$-$11	9.377169e$-$05	0.010797
hs95	30	155	1.561995e$-$02	2.167244e$-$11	1.255226e$-$06	0.006987
hs96	15	22	1.571116e$-$02	5.942702e$-$09	1.456985e$-$05	0.007057
hs97	23	45	4.071230e+00	5.430853e$-$08	7.079277e$-$05	0.000927
hs98	23	41	4.071231e+00	5.204100e$-$08	7.454991e$-$05	0.000917
hs99	8	9	$-$8.310799e+08	5.530622e$-$06	9.172217e$-$07	1.000000

Table 8. Table 8: Infeasible CUTEr hs test results, 116 successful cases out of 126 infeasible problems.

Problem	# iter	# $f$	$f\left(x^{\ast }\right)$	$v\left(x^{\ast }\right)$	KKT	Final $\rho$
hs100_inf	1	2	7.050369e+02	1.000000e+00	2.737830e$-$06	1.368915e$-$07
hs100lnp_inf	12	27	6.962388e+02	1.000322e+00	1.038439e$-$08	9.697737e$-$03
hs100mod_inf	1	2	7.050369e+02	1.000000e+00	2.737830e$-$06	1.368915e$-$07
hs101_inf	41	117	2.962431e+03	1.137023e+00	1.519645e$-$02	1.213144e$-$06
hs102_inf	45	103	2.998736e+03	1.031044e+00	5.193474e$-$03	3.135696e$-$05
hs103_inf	46	131	2.786811e+03	1.000003e+00	3.274512e$-$06	1.955742e$-$05
hs104_inf	22	33	4.200000e+00	1.011048e+00	1.779853e$-$06	3.311568e$-$03
hs105_inf	2	5	1.170198e+03	1.000000e+00	0.000000e+00	3.866220e$-$08
hs106_inf	55	59	2.000000e+03	2.275122e+00	9.272761e$-$04	1.642320e$-$02
hs107_inf	29	55	5.055009e+03	1.000001e+00	4.884375e$-$07	1.559948e$-$04
hs108_inf	20	25	$-$8.660197e$-$01	1.000002e+00	1.321342e$-$06	5.000000e$-$01
hs109_inf	64	140	4.777502e+03	3.656297e+02	3.723947e$-$02	6.828624e$-$03
hs10_inf	10	45	$-$1.000000e+00	1.000000e+00	4.253509e$-$11	5.000000e$-$01
hs111_inf	31	42	$-$4.529062e+01	1.000000e+00	1.557059e$-$07	4.458838e$-$02
hs111lnp_inf	23	101	$-$4.529060e+01	1.000001e+00	1.103729e$-$07	1.079561e$-$02
hs112_inf	24	25	$-$4.776110e+01	1.000002e+00	1.159142e$-$06	4.198856e$-$02
hs113_inf	22	48	4.240308e+01	1.000000e+00	1.803741e$-$06	3.380110e$-$02
hs114_inf	129	137	$-$8.509095e+02	6.615003e+02	1.557715e+00	2.503156e$-$02
hs117_inf	1	2	2.398758e+03	1.000000e+00	8.926099e$-$06	1.368915e$-$07
hs118_inf	23	24	9.094002e+02	1.000002e+00	7.007491e$-$07	2.084545e$-$02
hs119_inf	16	17	2.450109e+02	1.000057e+00	1.161125e$-$08	2.431977e$-$08
hs11_inf	9	10	$-$7.998667e+00	1.000000e+00	2.613585e$-$08	1.902388e$-$01
hs12_inf	1	2	$-$7.451564e$-$02	1.000000e+00	0.000000e+00	1.368915e$-$07
hs13_inf	5	6	1.932833e+00	1.000000e+00	0.000000e+00	1.215767e$-$01
hs14_inf	24	55	1.393462e+00	1.000002e+00	1.634047e$-$06	3.695115e$-$01
hs15_inf	4	5	4.657848e$-$01	1.000000e+00	1.668519e$-$06	9.697737e$-$03
hs16_inf	6	7	6.417104e$-$01	1.000000e+00	0.000000e+00	2.906321e$-$06
hs17_inf	33	829	2.052191e+00	1.000000e+00	8.590665e$-$07	1.405123e$-$10
hs18_inf	55	85	1.568117e+02	2.000000e+00	6.661338e$-$16	9.740833e$-$04
hs1_inf	4	5	1.258025e+01	1.000000e+00	0.000000e+00	8.709974e$-$10
hs20_inf	11	12	1.178896e+02	1.000000e+00	0.000000e+00	1.083106e$-$09
hs21_inf	3	68	$-$9.999000e+01	1.000000e+00	0.000000e+00	7.504732e$-$02
hs21mod_inf	7	8	$-$9.598670e+01	1.000000e+00	1.351003e$-$14	1.781441e$-$01
hs22_inf	5	40	1.000009e+00	1.000000e+00	1.623803e$-$08	5.000000e$-$01
hs23_inf	18	20	1.999963e+00	1.000019e+00	1.664523e$-$05	3.109002e$-$01
hs24_inf	1	2	$-$1.336948e$-$02	1.000000e+00	1.626221e$-$09	1.368915e$-$07
hs25_inf	10	13	3.283500e+01	1.000000e+00	7.749357e$-$14	1.000000e+00
hs268_inf	2	4	3.180734e+03	1.000000e+00	1.412204e$-$13	1.873928e$-$14
hs26_inf	13	36	6.949105e$-$02	1.000000e+00	4.654743e$-$10	1.213261e$-$08
hs27_inf	5	7	9.265421e$-$02	2.000000e+00	3.150539e$-$14	1.232023e$-$07
hs28_inf	2	3	5.001065e$-$08	1.000000e+00	8.255296e$-$11	1.368915e$-$07
hs29_inf	1	2	$-$1.001498e+00	1.000000e+00	1.567583e$-$07	1.368915e$-$07
hs2_inf	11	47	2.500000e+01	1.000000e+00	1.756152e$-$10	1.901548e$-$12
hs30_inf	1	2	2.990881e+00	1.000000e+00	0.000000e+00	1.368915e$-$07
hs31_inf	28	527	1.873971e+01	1.000000e+00	8.464849e$-$07	6.844574e$-$08
hs32_inf	1	2	7.049285e+00	1.000000e+00	1.216813e$-$06	1.368915e$-$07
hs33_inf	1	2	$-$3.000761e+00	1.000000e+00	1.673118e$-$06	1.368915e$-$07
hs34_inf	1	2	$-$7.605082e$-$04	1.000000e+00	0.000000e+00	1.368915e$-$07
hs35_inf	1	2	2.228000e+00	1.000000e+00	0.000000e+00	1.368915e$-$07
hs35i_inf	1	2	2.228000e+00	1.000000e+00	0.000000e+00	1.368915e$-$07
hs35mod_inf	1	2	2.234820e+00	1.000000e+00	4.563049e$-$07	1.368915e$-$07
hs36_inf	6	7	$-$2.634143e+02	1.000000e+00	1.519082e$-$08	1.815123e$-$10
hs37_inf	7	76	$-$3.150257e+02	1.000000e+00	3.474836e$-$16	2.144845e$-$06
hs38_inf	5	6	2.801498e+01	1.000000e+00	0.000000e+00	3.807341e$-$09
hs3_inf	10	59	$-$2.846398e$-$14	1.000000e+00	1.907600e$-$10	1.000000e+00
hs3mod_inf	2	3	9.983686e$-$01	1.000000e+00	0.000000e+00	1.368915e$-$07
hs40_inf	23	24	$-$2.500005e$-$01	1.000003e+00	1.605618e$-$06	1.000000e+00
hs41_inf	4	5	1.950869e+00	1.000000e+00	9.205041e$-$09	6.026930e$-$08
hs42_inf	1	13	1.400000e+01	2.000000e+00	7.605085e$-$07	1.368915e$-$07
hs43_inf	1	2	$-$4.101071e$-$01	1.000000e+00	0.000000e+00	1.368915e$-$07
hs44_inf	1	2	$-$1.520438e$-$03	1.000000e+00	1.521016e$-$07	1.368915e$-$07
hs44new_inf	1	2	$-$1.002280e+00	1.000000e+00	0.000000e+00	1.368915e$-$07
hs45_inf	3	4	1.445187e+00	1.000000e+00	6.341053e$-$11	5.143153e$-$10
hs46_inf	2	9	3.277487e+00	1.000000e+00	1.715330e$-$07	1.475042e$-$09
hs47_inf	12	24	2.172792e$-$01	1.000040e+00	4.105068e$-$09	1.638935e$-$09
hs48_inf	18	178	7.951999e$-$06	1.000002e+00	1.978232e$-$06	1.711143e$-$08
hs49_inf	7	8	7.144037e$-$03	1.000001e+00	8.551244e$-$09	1.368915e$-$07
hs4_inf	2	3	6.265554e$-$01	1.000000e+00	1.521016e$-$07	1.368915e$-$07
hs50_inf	10	11	4.392824e$-$05	1.011312e+00	6.405816e$-$07	9.000000e$-$01
hs51_inf	20	49	1.048914e$-$10	1.000006e+00	2.581146e$-$06	3.125000e$-$02
hs52_inf	22	23	5.499964e+00	1.000006e+00	2.523975e$-$06	2.257227e$-$02
hs53_inf	22	23	5.499973e+00	1.000005e+00	1.457471e$-$06	4.416682e$-$02
hs54_inf	6	8	$-$2.282916e$-$90	1.000000e+00	4.000073e$-$04	1.000000e+00
hs55_inf	13	14	6.666659e+00	1.000053e+00	4.316596e$-$06	1.119802e$-$04
hs56_inf	7	13	$-$2.379995e+00	1.000382e+00	3.469250e$-$08	3.138106e$-$01
hs57_inf	1	2	5.079755e$-$02	4.000000e$-$01	4.024329e$-$06	1.368915e$-$07
hs59_inf	50	125	5.534983e+00	1.076927e+01	4.356665e$-$05	7.221198e$-$02
hs5_inf	1	2	9.905029e$-$01	1.000000e+00	0.000000e+00	1.368915e$-$07
hs60_inf	11	43	7.914655e$-$02	1.000000e+00	2.996201e$-$07	1.368915e$-$07
hs61_inf	22	32	$-$7.189256e+01	2.750853e+00	2.527758e$-$07	6.438145e$-$11
hs62_inf	2	38	$-$2.569993e+04	1.000000e+00	1.712570e$-$19	6.844574e$-$08
hs63_inf	17	23	9.681069e+02	1.000113e+00	1.265109e$-$06	1.041756e$-$01
hs65_inf	3	4	6.400195e+00	1.000000e+00	3.431161e$-$06	4.295800e$-$08
hs66_inf	8	118	5.183511e$-$01	1.000000e+00	1.917211e$-$06	5.978711e$-$02
hs67_inf	49	55	$-$5.179534e$-$01	1.000000e+00	0.000000e+00	1.930302e$-$08
hs68_inf	4	5	$-$8.692912e$-$01	1.010548e+00	7.077349e$-$07	3.486784e$-$01
hs69_inf	10	50	$-$9.433878e+02	1.000000e+00	3.979935e$-$08	1.933110e$-$08
hs6_inf	9	18	2.632899e$-$08	1.000000e+00	2.995589e$-$08	1.368915e$-$07
hs70_inf	3	4	2.133158e+00	1.000000e+00	5.329050e$-$08	5.892729e$-$08
hs71_inf	20	27	1.684920e+01	1.000022e+00	2.198781e$-$05	6.250000e$-$02
hs73_inf	12	13	2.996091e+01	1.000000e+00	2.680150e$-$15	1.230964e$-$02
hs74_inf	11	12	3.555000e+03	4.793332e+02	1.295432e$-$03	1.039186e$-$01
hs75_inf	10	11	3.555000e+03	4.793329e+02	9.790964e$-$04	1.105956e$-$01
hs76_inf	1	2	$-$1.257980e+00	1.000000e+00	0.000000e+00	1.368915e$-$07
hs76i_inf	1	2	$-$1.257980e+00	1.000000e+00	0.000000e+00	1.368915e$-$07
hs77_inf	53	180	6.956721e$-$01	1.000268e+00	2.285362e$-$05	1.868347e$-$03
hs79_inf	18	19	1.504972e$-$01	1.000012e+00	5.587084e$-$06	2.058911e$-$01
hs7_inf	8	9	$-$1.732051e+00	1.000000e+00	5.038447e$-$09	1.000000e+00
hs81_inf	32	70	1.000008e+00	1.000009e+00	9.151413e$-$06	3.223196e$-$01
hs83_inf	17	18	$-$2.986759e+04	1.415960e+01	7.181688e$-$07	1.049888e$-$03
hs85_inf	47	48	4.137154e+01	9.368586e+06	8.784126e$-$06	1.102048e$-$03
hs86_inf	1	2	1.224387e+01	1.000000e+00	5.627761e$-$06	1.368915e$-$07
hs87_inf	10	11	9.347058e+03	1.000208e+00	8.220663e$-$07	4.212087e$-$03
hs88_inf	5	6	1.169516e+00	1.000399e+00	1.011916e$-$07	1.232023e$-$07
hs89_inf	22	148	5.024977e+00	1.051266e+00	4.016240e$-$08	2.248197e$-$09
hs8_inf	13	15	$-$1.000000e+00	1.955966e+00	3.366090e$-$06	1.000000e+00
hs90_inf	13	29	1.543613e+00	1.000000e+00	0.000000e+00	4.230379e$-$09
hs91_inf	12	39	1.842669e+00	1.000000e+00	0.000000e+00	3.426607e$-$09
hs92_inf	10	78	1.823176e+00	1.000000e+00	0.000000e+00	1.368915e$-$07
hs93_inf	5	14	1.692649e$-$14	3.070000e+00	5.126290e$-$21	2.996907e$-$04
hs95_inf	8	11	1.292305e+00	6.907751e$-$01	1.852601e$-$03	6.953210e$-$03
hs96_inf	6	8	$-$1.137676e$-$01	1.002112e+00	2.102580e$-$03	1.126420e$-$02
hs97_inf	17	26	3.654643e+00	1.001483e+00	8.537328e$-$04	2.783380e$-$05
hs98_inf	14	23	3.779482e+00	1.003179e+00	2.049170e$-$03	3.672281e$-$05
hs99_inf	10	13	$-$8.310797e+08	1.004413e+00	4.413353e$-$03	1.000000e+00
hs9_inf	1	19	1.439486e$-$10	1.000000e+00	1.710708e$-$07	5.983858e$-$07

Equations

$${\mathrm{dist}}_2\left(y^i|C_i\right):=\underset{z^i\in C_i}{\inf }\Vert y^i-z^i{\Vert }_2.$$

$$y=\zeta a\text{with}\zeta =\frac{1}{\Vert a{\Vert }_2^2}⟨y,a⟩,\text{meaning that}{\delta }^{\ast }(y|C)=-\zeta b.$$

$$\underset{x\in {\mathbb{R}}^n}{\min }$$

$$s.t.$$

$$c_i(x)\le 0\text{for all}i\in \{\overline{m}+1,\dots ,m\}.$$

$$v(x)=\sum _{i=1}^{\overline{m}}\left|c_i(x)\right|+\sum _{i=\overline{m}+1}^m(c_i(x){)}_{+}\text{and}\varphi (x,\rho )=\rho f(x)+v(x).$$

$$\underset{x\in {\mathbb{R}}^n}{\min }\varphi (x,0),\text{where}\varphi (x,0)=v(x).$$

$$F(x,\rho ,\eta )=\rho f(x)+⟨\eta ,c(x)⟩.$$

$$J(d,\rho ;x^k,{\eta }^k):=l(d,\rho ;x^k)+\frac{1}{2}⟨d,H(\rho ;x^k,{\eta }^k)d⟩+\delta (d|X),$$

$$l(d,\rho ;x^k)=\rho ⟨\nabla f(x^k),d⟩+\sum _{i=1}^{\overline{m}}\left|c_i(x^k)+⟨\nabla c_i(x^k),d⟩\right|+\sum _{i=\overline{m}+1}^m(c_i(x^k)+⟨\nabla c_i(x^k),d⟩{)}_{+}$$

$$H(\rho ;x^k,{\eta }^k)\approx {\nabla }_{xx}^{2}F(\rho ;x^k,{\eta }^k)=\rho {\nabla }_{xx}^{2}f(x^k)+\sum _{i=1}^m{\eta }_i^k{\nabla }_{xx}^2{c}_i(x^k).$$

$$d^k\approx \arg \underset{d\in {\mathbb{R}}^n}{\min }J(d,{\rho }_k;x^k,{\eta }^k)\text{for some}{\rho }_k\in (0,{\rho }_{k-1}].$$

$$\Delta l(d^k,0;x^k)$$

$$\text{where generally}\Delta l(d^k,{\rho }_k;x^k)$$

$$\Delta J(d^k,{\rho }_k;x^k,{\eta }^k):=J(0,{\rho }_k;x^k,{\eta }^k)-J(d^k,{\rho }_k;x^k,{\eta }^k),$$

$$\varphi (x^k+{\alpha }_k{d}^k,{\rho }_k)-\varphi (x^k,{\rho }_k)\le -\theta {\alpha }_k\Delta l(d^k,{\rho }_k;x^k).$$

$$g=\nabla f(x^k),a^i=\nabla c_i(x^k),b_i=c_i(x^k),A=[a^1,\cdots ,a^m{]}^T,$$

$$H_f\approx {\nabla }_{xx}^{2}f(x^k),H_0\approx \sum _{i=1}^m{\eta }_i^k{\nabla }_{xx}^2{c}_i(x^k),\text{and}{H}_{\rho }=\rho H_f+H_0.$$

$$C_i$$

$$\text{and}{C}_i$$

$$\psi (d,\rho )=\rho ⟨g,d⟩+\frac{1}{2}⟨d,H_{\rho }d⟩\text{and}l(d,0)=\sum _{i=1}^m\Vert a^i{\Vert }_2{\mathrm{dist}}_2\left(d|C_i\right),$$

$$\underset{d\in {\mathbb{R}}^n}{\min }J(d,\rho ),\text{where}J(d,\rho )=\psi (d,\rho )+l(d,0)+\delta (d|X).$$

$$\underset{\mathbf{u}\in {\mathbb{R}}^n\times \cdots \times {\mathbb{R}}^n}{\max }D(\mathbf{u},\rho )s.t.$$

$$u^i\in {\mathbb{B}}_2\text{for all}i\in \{1,\dots ,m\},$$

$$D(\mathbf{u},\rho )=-\frac{1}{2}⟨u^0-\rho g,H_{\rho }^{-1}(u^0-\rho g)⟩-\sum _{i=1}^m\Vert a^i{\Vert }_2{\delta }^{\ast }(u^i|C_i)-{\delta }^{\ast }(u^{m+1}|X).$$

$$u^i={\zeta }_i(\mathbf{u}){\bar{a}}^i,$$

$${\zeta }_i(\mathbf{u})\in \{\begin{array}{ll}[-1,1]& \text{for all i∈{1,...,mˉ}}\\ [0,1]& \text{for all i∈{mˉ+1,...,m}},\end{array}$$

$${\delta }^{\ast }(u^i|C_i)=-{\zeta }_i(\mathbf{u}){\bar{b}}_i.$$

$$D(\mathbf{u},0)\le J(d,0)\text{and}D(\mathbf{u},\rho )\le J(d,\rho ).$$

$$P_{C_i}(y^i):=\arg \underset{z^i\in C_i}{\min }{\Vert z^i-y^i\Vert }_2$$

$$P_{C_i}(d)=$$

$$J(d^{(j)},{\rho }_{(j)})\le J(0,{\rho }_{(j)}).$$

$$D({\mathbf{w}}^{(j)},0)\ge D({\mathbf{u}}^{(j)},0)\ge D({\mathbf{u}}^{(0)},0)>-\infty .$$

$$J^{(0)}:=J(0,\rho )=J(0,0)=l(0,0)=\sum _{i=1}^{\bar{m}}|b_i|+\sum _{i=\bar{m}+1}^m(b_i{)}_{+}\ge 0.$$

Full text

{AMS}

49M20, 49M29, 49M37, 65K05, 65K10, 90C06, 90C20, 90C25

1 Introduction

In this paper, we consider the use of sequential quadratic optimization (commonly known as SQP) methods for solving large-scale nonlinear optimization problems (NLPs) [1, 2, 3, 4, 9, 15, 18]. While they have proved to be effective for solving small- to medium-scale problems, SQP methods have traditionally faltered in large-scale settings due to the expense of (accurately) solving large-scale quadratic subproblems (QPs) during each iteration. However, with the use of matrix-free methods for solving these subproblems, one may consider the acceptance of inexact subproblem solutions. Such a feature offers the possibility of terminating the subproblem solver early, perhaps well before an accurate solution has been computed. This characterizes the type of strategy that we propose in this paper.

Some work has been done to provide global convergence guarantees for SQP methods that allow inexact subproblem solves [8]. However, the practical efficiency of such an approach remains an open question. A critical aspect of their implementation is the choice of subproblem solver since it must be able to provide good inexact solutions quickly, as well as have the ability to compute highly accurate solutions—say, by exploiting well-chosen starting points—in the neighborhood of a solution of the NLP. In addition, while a global convergence mechanism such as a merit function or filter is necessary to guarantee convergence from remote starting points, any NLP algorithm can suffer when such a mechanism does not immediately guide the algorithm toward promising regions of the search space. To confront this issue when an exact penalty function is used as a merit function, we propose a dynamic penalty parameter updating strategy to be incorporated within the subproblem solver so that each computed search direction predicts progress toward both feasibility and optimality. This strategy represents a stark contrast to previously proposed techniques that only update the penalty parameter after a sequence of iterations, in hindsight at the end of an iteration [1, 9, 10], or at the expense of numerous subproblem solves within a single iteration [3, 5, 7].

To provide some context about how the algorithm proposed in this paper compares to other recently proposed SQP-type methods in the literature, let us contrast our approach with those proposed in [3] and [8]. The penalty SQP method proposed in [3] was motivated by the desire to formulate an SQP approach that attains strong convergence guarantees when solving problems regardless of whether they involve constraints that are feasible or infeasible. Toward this end, the approach involved a novel dynamic updating scheme for the penalty parameter that, e.g., quickly drives the algorithm toward constraint violation minimization when infeasibility is detected. The approach relies on exact solves of two QP subproblems per iteration; the first determines the reduction that can be obtained in a local model of an infeasibility measure while the second minimizes a local model of the objective while ensuring that the reduction in a local model of the infeasibility measure is proportional to that attained by the solution of the first QP. In this manner, rapid convergence can be attained when solving either a feasible or infeasible problem, although a high price is paid by needing exact subproblem solutions. The method in [8] overcomes this obstacle by allowing inexact subproblem solves. However, it also potentially requires (approximate) solutions of two QPs per iteration, one aimed at minimizing constraint violation and one aimed at reducing the objective subject to an appropriate bound on constraint violation. The approach proposed in this paper also allows inexactness in the QP solves, but only requires solving a single QP in each iteration. This is made possible by a new strategy for dynamically updating the penalty parameter within the QP solver. This dynamic penalty parameter updating strategy is the focus of our investigation. We prove that our algorithm does not reduce the penalty parameter unnecessarily and that one can ensure convergence to an optimal solution (when a given problem is feasible) or to an infeasible stationary point (when a given problem is infeasible).

Overall, the contributions in this paper can be summarized as the following.

• •

Our proposed SQP technique is specifically designed to be effective in large-scale settings. In particular, it allows for the use of iterative methods for solving the QP subproblems, allowing inexactness in the subproblem solves.

• •

Our technique involves a dynamic penalty parameter updating strategy to be employed within the subproblem solve. This makes the approach efficient while not having to accurately solve multiple QPs in a single iteration.

• •

By ensuring that each computed step predicts progress toward minimizing constraint violation, our technique allows for automatic infeasibility detection.

1.1 Organization

In the remainder of this section, we outline our notation and introduce various concepts that will be employed throughout the paper. In §2, we introduce a basic penalty-SQP algorithm. Our penalty parameter updating strategy is detailed in §3. A complete algorithm is presented and analyzed in §4. The results of numerical experiments are presented in §6. Concluding remarks are provided in §7.

1.2 Notation

Let $\mathbb{R}^{n}$ be the space of real $n$-vectors, $\mathbb{R}^{n}_{+}$ be the nonnegative orthant of $\mathbb{R}^{n}$ (i.e., $\mathbb{R}^{n}_{+}:=\{x\in\mathbb{R}^{n}:x\geq 0\}$), and $$