# Generalized Additive Modeling of Ecological Data With mgcv: New Adequacy Assessment Tools

## TL;DR

This paper introduces new tools for assessing the statistical adequacy of generalized additive models (GAMs) in ecological data analysis using the mgcv package in R.

## Contribution

The paper presents new helper functions for the hnp package and a metric from mgcViz to evaluate GAM model adequacy and detect under- or overfitting.

## Key findings

- Half-normal plots with simulated envelopes help validate GAMs against distributional assumptions.
- A new metric from mgcViz aids in detecting under- or overfitting in GAMs.
- Three fisheries examples demonstrate the utility of these tools for analyzing ecological nonlinearities.

## Abstract

Generalized additive models (GAMs) are a semi‐parametric extension of generalized linear models (GLMs) that allow incorporating different forms of nonlinearities commonly encountered in ecological relationships, thus frequently offering a better statistical description than GLMs in such cases. Due to the use of smooth functions, however, validating that the observed data represent a plausible realization of a fitted GAM according to the underlying distributional assumptions being used is less straightforward than with GLMs. Moreover, if the number of basis dimensions used in smooth terms to control the degree of flexibility is set too large, overfitting can arise despite in‐built penalization procedures aimed at preventing excessive wiggliness. Here, we present how GAMs fitted with the mgcv package in R can be assessed for their adequacy based on half‐normal plots with a simulated envelope using newly‐available helper functions for the hnp package. A proposed metric relying on the mgcViz package is also presented to help detect both under‐ and overfitting relative to a predictor of interest from a realized coverage perspective. Three fisheries‐related examples analyzing continuous data, counts, and discrete proportions are then presented to illustrate the usefulness of these approaches in providing more statistical context for the interpretation of nonlinear ecological relationships.

Anadromous Nunavik Arctic charr (
Salvelinus alpinus
) during the upstream migration in the Aipparusik (Bérard) River in September 2017, Tasiujaq, Québec, Canada. Sampled female Arctic charr from this population and two other populations in Nunavik were jointly analyzed to estimate their length‐at‐maturity. This was achieved with the analysis of discrete proportions (i.e., number of females exhibiting developing gonads within 50‐mm fork length bins) using the gam() function of the mgcv package. The adequacy assessment tools presented in this study were applied to this real‐case example (photo: Pascal Ouellet, MELCCFP).