Improved boosting algorithms using confidence-rated predictions

Robert E. Schapire, Yoram Singer

Research output: Contribution to journalConference article

2094 Scopus citations

Abstract

We describe several improvements to Freund and Schapire's AdaBoost boosting algorithm, particularly in a setting in which hypotheses may assign confidences to each of their predictions. We give a simplified analysis of AdaBoost in this setting, and we show how this analysis can be used to find improved parameter settings as well as a refined criterion for training weak hypotheses. We give a specific method for assigning confidences to the predictions of decision trees, a method closely related to one used by Quinlan. This method also suggests a technique for growing decision trees which turns out to be identical to one proposed by Kearns and Mansour. We focus next on how to apply the new boosting algorithms to multiclass classification problems, particularly to the multi-label case in which each example may belong to more than one class. We give two boosting methods for this problem, plus a third method based on output coding. One of these leads to a new method for handling the single-label case which is simpler but as effective as techniques suggested by Freund and Schapire. Finally, we give some experimental results comparing a few of the algorithms discussed in this paper.

Original languageEnglish (US)
Pages (from-to)297-336
Number of pages40
JournalMachine Learning
Volume37
Issue number3
DOIs
StatePublished - Dec 1999
EventProceedings of the 1998 11th Annual Conference on 'Computational Learning Theory' (COLT98) - Madison, WI, USA
Duration: Jul 24 1998Jul 26 1998

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Improved boosting algorithms using confidence-rated predictions'. Together they form a unique fingerprint.

  • Cite this