New confidence interval methods for Shannon index

TL;DR

This paper introduces and compares two new methods for constructing confidence intervals for Shannon's diversity index, demonstrating their effectiveness through simulations and real data application, with the bootstrap-t method showing superior performance.

Contribution

The paper proposes two novel confidence interval methods for Shannon's index and compares their performance with existing techniques using extensive simulations.

Findings

Bootstrap-t method achieved the best coverage percentage.

Both proposed methods are feasible for estimating Shannon's diversity.

High performance observed in high dominance community scenarios.

Abstract

Several factors affect the structure of communities, including biological, physical and chemical phenomena, impacting the quantification of biodiversity, measured by diversity indexes such as Shannon's entropy. Then, once a point estimate is obtained, confidence intervals methods such as the bootstrap ones are often used. These methods, however, can have different performances, which many authors have revealed in the last decade. Furthermore, problems such as the asymmetry of the distribution of estimates and the possibility of Shannon's diversity index estimator bias can lead to incorrect recommendations to the research community. Thus, we propose two methods and compare them with seven others using their performances to face these problems. The first idea uses the credible interval (CI) method to build a bootstrap confidence interval. The second one starts by correcting the bias and then uses an asymptotic approach. We considered 27 community structures representing scenarios with high dominance, high codominance or moderate dominance, the number of species equal to 4, 20 or 80 and 10, 50 or 500 individuals to compare their performances. Then, we generated 1000 samples, built 95% confidence intervals, and calculated the percentage of times they included the community diversity index (coverage percentage) for each community structure. Our results showed the feasibility of both proposed methods to estimate Shannon's diversity. The simulation study revealed the bootstrap-t technique had the best performance, i.e., best coverage percentage, compared with the other methods. Finally, we illustrate the methodology by applying it to an original aphid and parasitoid species dataset. We recommend the bootstrap-t when the community structure analysed is similar to the simulated ones. Also, the methods provided high performance for the high dominance scenarios.