Newton-type Methods for REML Estimation in Genetic Analysis of Quantitative Traits

TL;DR

This paper introduces improved Newton-type optimization methods for REML estimation in genetic analysis, addressing boundary issues and enhancing robustness and efficiency in variance component estimation.

Contribution

It proposes new approaches that incorporate constraints into Newton methods, using average information and BFGS Hessian approximations for better REML optimization.

Findings

Enhanced robustness in boundary cases

Improved efficiency over standard methods

Validated on real animal population data

Abstract

Robust and efficient optimization methods for variance component estimation using Restricted Maximum Likelihood (REML) models for genetic mapping of quantitative traits are considered. We show that the standard Newton-AI scheme may fail when the optimum is located at one of the constraint boundaries, and we introduce different approaches to remedy this by taking the constraints into account. We approximate the Hessian of the objective function using the average information matrix and also by using an inverse BFGS formula. The robustness and efficiency is evaluated for problems derived from two experimental data from the same animal populations.