### Abstract

We consider the problem of estimating the probability of an observed string drawn i.i.d. from an unknown distribution. The key feature of our study is that the length of the observed string is assumed to be of the same order as the size of the underlying alphabet. In this setting, many letters are unseen and the empirical distribution tends to overestimate the probability of the observed letters. To overcome this problem, the traditional approach to probability estimation is to use the classical Good-Turing estimator. We introduce a natural scaling model and use it to show that the Good-Turing sequence probability estimator is not consistent. We then introduce a novel sequence probability estimator that is indeed consistent under the natural scaling model.

Original language | English (US) |
---|---|

Title of host publication | Proceedings - 2007 IEEE International Symposium on Information Theory, ISIT 2007 |

Pages | 2356-2360 |

Number of pages | 5 |

DOIs | |

State | Published - Dec 1 2007 |

Event | 2007 IEEE International Symposium on Information Theory, ISIT 2007 - Nice, France Duration: Jun 24 2007 → Jun 29 2007 |

### Publication series

Name | IEEE International Symposium on Information Theory - Proceedings |
---|---|

ISSN (Print) | 2157-8101 |

### Other

Other | 2007 IEEE International Symposium on Information Theory, ISIT 2007 |
---|---|

Country | France |

City | Nice |

Period | 6/24/07 → 6/29/07 |

### All Science Journal Classification (ASJC) codes

- Theoretical Computer Science
- Information Systems
- Modeling and Simulation
- Applied Mathematics

## Fingerprint Dive into the research topics of 'A better good-turing estimator for sequence probabilities'. Together they form a unique fingerprint.

## Cite this

*Proceedings - 2007 IEEE International Symposium on Information Theory, ISIT 2007*(pp. 2356-2360). [4557571] (IEEE International Symposium on Information Theory - Proceedings). https://doi.org/10.1109/ISIT.2007.4557571