An information-pattern-based approach to novelty detection

Xiaoyan Li, W. Bruce Croft

Research output: Contribution to journalArticlepeer-review

23 Scopus citations

Abstract

In this paper, a new novelty detection approach based on the identification of sentence level information patterns is proposed. First, "novelty" is redefined based on the proposed information patterns, and several different types of information patterns are given corresponding to different types of users' information needs. Second, a thorough analysis of sentence level information patterns is elaborated using data from the TREC novelty tracks, including sentence lengths, named entities (NEs), and sentence level opinion patterns. Finally, a unified information-pattern-based approach to novelty detection (ip-BAND) is presented for both specific NE topics and more general topics. Experiments on novelty detection on data from the TREC 2002, 2003 and 2004 novelty tracks show that the proposed approach significantly improves the performance of novelty detection in terms of precision at top ranks. Future research directions are suggested.

Original languageEnglish (US)
Pages (from-to)1159-1188
Number of pages30
JournalInformation Processing and Management
Volume44
Issue number3
DOIs
StatePublished - May 2008
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Media Technology
  • Computer Science Applications
  • Management Science and Operations Research
  • Library and Information Sciences

Keywords

  • Information patterns
  • Information retrieval
  • Named entities
  • Novelty detection
  • Question answering

Fingerprint

Dive into the research topics of 'An information-pattern-based approach to novelty detection'. Together they form a unique fingerprint.

Cite this