A new universal estimator of divergence is presented for multidimensional continuous densities based on κ-nearest-neighbor (κ-NN) distances. Assuming independent and identically distributed (i.i.d.) samples, the new estimator is proved to be asymptotically unbiased and mean-square consistent. In experiments with high-dimensional data, the κ-NN approach generally exhibits faster convergence than previous algorithms. It is also shown that the speed of convergence of the κ-NN method can be further improved by an adaptive choice of κ.
All Science Journal Classification (ASJC) codes
- Information Systems
- Computer Science Applications
- Library and Information Sciences
- Information measure
- Random vector
- Universal estimation