Penalized empirical likelihood estimation and EM algorithms for closed-population capture-recapture models

TL;DR

This paper extends empirical likelihood methods for capture-recapture models to include behavioral effects, introduces a penalized approach to improve stability, and develops EM algorithms for practical estimation, demonstrating improved reliability through simulations and real data.

Contribution

The paper introduces a penalized empirical likelihood method for capture-recapture models with behavioral effects and develops EM algorithms to enhance estimation stability and performance.

Findings

PEL overcomes EL instability in low capture probability scenarios

EM algorithms improve practical estimation of PEL and EL methods

Simulation and real data show PEL and EM outperform existing methods

Abstract

Capture-recapture experiments are widely used to estimate the abundance of a finite population. Based on capture-recapture data, the empirical likelihood (EL) method has been shown to outperform the conventional conditional likelihood (CL) method. However, the current literature on EL abundance estimation ignores behavioral effects, and the EL estimates may not be stable, especially when the capture probability is low. We make three contributions in this paper. First, we extend the EL method to capture-recapture models that account for behavioral effects. Second, to overcome the instability of the EL method, we propose a penalized EL (PEL) estimation method that penalizes large abundance values. We then investigate the asymptotics of the maximum PEL estimator and the PEL ratio statistic. Third, we develop standard expectation-maximization (EM) algorithms for PEL to improve its practical performance. The EM algorithm is also applicable to EL and CL with slight modifications. Our simulation and a real-world data analysis demonstrate that the PEL method successfully overcomes the instability of the EL method and the proposed EM algorithm produces more reliable results than existing optimization algorithms.