Analysis of KNN Density Estimation
Puning Zhao, Lifeng Lai

TL;DR
This paper provides a detailed analysis of the convergence rates of kNN density estimation under different support conditions, establishing minimax optimality in certain cases and comparing with kernel methods.
Contribution
It offers the first comprehensive convergence rate analysis of kNN density estimation for both bounded and unbounded support sets, including minimax optimality results.
Findings
kNN density estimation is minimax optimal with known bounded support.
For unknown support, $ ext{l}_1$ error remains optimal, but $ ext{l}_ ext{infinity}$ error does not converge.
In unbounded support, $ ext{l}_ ext{infinity}$ error is nearly minimax optimal, outperforming kernel density estimation.
Abstract
We analyze the and convergence rates of k nearest neighbor density estimation method. Our analysis includes two different cases depending on whether the support set is bounded or not. In the first case, the probability density function has a bounded support and is bounded away from zero. We show that kNN density estimation is minimax optimal under both and criteria, if the support set is known. If the support set is unknown, then the convergence rate of error is not affected, while error does not converge. In the second case, the probability density function can approach zero and is smooth everywhere. Moreover, the Hessian is assumed to decay with the density values. For this case, our result shows that the error of kNN density estimation is nearly minimax optimal. The error does not reach the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
