Effects of sample size on the performance of species distribution models

M. S. Wisz, R. J. Hijmans, J. Li, A. T. Peterson, C. H. Graham, A. Guisan, J. Elith, M. Dudík, S. Ferrier, F. Huettmann, J. R. Leathwick, A. Lehmann, L. Lohmann, B. A. Loiselle, G. Manion, C. Moritz, M. Nakamura, Y. Nakazawa, J. Mc C. Overton, S. J. PhillipsK. S. Richardson, R. Scachetti-Pereira, R. E. Schapire, J. Soberón, S. E. Williams, N. E. Zimmermann

Research output: Contribution to journalArticle

1166 Scopus citations

Abstract

A wide range of modelling algorithms is used by ecologists, conservation practitioners, and others to predict species ranges from point locality data. Unfortunately, the amount of data available is limited for many taxa and regions, making it essential to quantify the sensitivity of these algorithms to sample size. This is the first study to address this need by rigorously evaluating a broad suite of algorithms with independent presence-absence data from multiple species and regions. We evaluated predictions from 12 algorithms for 46 species (from six different regions of the world) at three sample sizes (100, 30, and 10 records). We used data from natural history collections to run the models, and evaluated the quality of model predictions with area under the receiver operating characteristic curve (AUC). With decreasing sample size, model accuracy decreased and variability increased across species and between models. Novel modelling methods that incorporate both interactions between predictor variables and complex response shapes (i.e. GBM, MARS-INT, BRUTO) performed better than most methods at large sample sizes but not at the smallest sample sizes. Other algorithms were much less sensitive to sample size, including an algorithm based on maximum entropy (MAXENT) that had among the best predictive power across all sample sizes. Relative to other algorithms, a distance metric algorithm (DOMAIN) and a genetic algorithm (OM-GARP) had intermediate performance at the largest sample size and among the best performance at the lowest sample size. No algorithm predicted consistently well with small sample size (n < 30) and this should encourage highly conservative use of predictions based on small sample size and restrict their use to exploratory modelling.

Original languageEnglish (US)
Pages (from-to)763-773
Number of pages11
JournalDiversity and Distributions
Volume14
Issue number5
DOIs
StatePublished - Sep 2008

All Science Journal Classification (ASJC) codes

  • Ecology, Evolution, Behavior and Systematics

Keywords

  • Ecological niche model
  • MAXENT
  • Model comparison
  • OM-GARP
  • Sample size
  • Species distribution model

Fingerprint Dive into the research topics of 'Effects of sample size on the performance of species distribution models'. Together they form a unique fingerprint.

  • Cite this

    Wisz, M. S., Hijmans, R. J., Li, J., Peterson, A. T., Graham, C. H., Guisan, A., Elith, J., Dudík, M., Ferrier, S., Huettmann, F., Leathwick, J. R., Lehmann, A., Lohmann, L., Loiselle, B. A., Manion, G., Moritz, C., Nakamura, M., Nakazawa, Y., Overton, J. M. C., ... Zimmermann, N. E. (2008). Effects of sample size on the performance of species distribution models. Diversity and Distributions, 14(5), 763-773. https://doi.org/10.1111/j.1472-4642.2008.00482.x