Effects of Influential Points and Sample Size on the Selection and Replicability of Multivariable Fractional Polynomial Models

TL;DR

This study examines how influential points and sample size affect the selection and replicability of multivariable fractional polynomial models, emphasizing diagnostic approaches and sample considerations for reliable modeling.

Contribution

It introduces methods to identify influential points impacting model selection and assesses the effects of sample size on model stability and replicability in MFP procedures.

Findings

Influential points can significantly alter model selection.

Small sample sizes may hinder detection of non-linear effects.

Careful diagnostics improve MFP model reliability.

Abstract

The multivariable fractional polynomial (MFP) procedure combines variable selection with a function selection procedure (FSP). For continuous variables, a closed test procedure is used to decide between no effect, linear, FP1 or FP2 functions. Influential observations (IPs) and small sample size can both have an impact on a selected fractional polynomial model. In this paper, we used simulated data with six continuous and four categorical predictors to illustrate approaches which can help to identify IPs with an influence on function selection and the MFP model. Approaches use leave-one or two-out and two related techniques for a multivariable assessment. In seven subsamples we also investigated the effects of sample size and model replicability. For better illustration, a structured profile was used to provide an overview of all analyses conducted. The results showed that one or more IPs can drive the functions and models selected. In addition, with a small sample size, MFP might not be able to detect non-linear functions and the selected model might differ substantially from the true underlying model. However, if the sample size is sufficient and regression diagnostics are carefully conducted, MFP can be a suitable approach to select variables and functional forms for continuous variables.