Estimating the distance at which narwhal $(\textit{Monodon monoceros})$ respond to disturbance: a penalized threshold hidden Markov model

TL;DR

This paper introduces a penalized hidden Markov model to accurately estimate behavioral response thresholds of narwhals to vessel disturbances, improving the assessment of disturbance impacts on marine mammals.

Contribution

The authors develop a novel lasso-penalized threshold hidden Markov model that efficiently estimates meaningful behavioral thresholds from telemetry data, addressing limitations of existing models.

Findings

Narwhals react to vessels up to 4 km away.

Narwhal behavior changes include decreased movement persistence.

Average maximum depth during response is 356 meters.

Abstract

Understanding behavioural responses to disturbances is vital for wildlife conservation. For example, in the Arctic, the decrease in sea ice has opened new shipping routes, increasing the need for impact assessments that quantify the distance at which marine mammals react to vessel presence. This information can then guide targeted mitigation policies, such as vessel slow-down regulations and delineation of avoidance areas. Using telemetry data to determine distances linked to deviations from normal behaviour requires advanced statistical models, such as threshold hidden Markov models (THMMs). While these are powerful tools, they do not assess whether the estimated threshold reflects a meaningful behavioural shift. We introduce a lasso-penalized THMM that builds on computationally efficient methods to impose penalties on HMMs and present a new, efficient penalized quasi-restricted maximum-likelihood estimator. Our framework is capable of estimating thresholds and assessing whether the disturbance effects are meaningful. With simulations, we demonstrate that our lasso method effectively shrinks spurious threshold effects towards zero. When applied to narwhal $\textit{(Monodon monoceros)}$ movement data, our analysis suggests that narwhal react to vessels up to 4 kilometres away by decreasing movement persistence and spending more time in deeper waters (average maximum depth of 356m). Overall, we provide a broadly applicable framework for quantifying behavioural responses to stimuli, with applications ranging from determining reaction thresholds to disturbance to estimating the distances at which terrestrial species, such as elephants, detect water.