A thorough examination of the CNN/daily mail reading comprehension task

Danqi Chen, Jason Bolton, Christopher D. Manning

Research output: Chapter in Book/Report/Conference proceedingConference contribution

155 Scopus citations

Abstract

Enabling a computer to understand a document so that it can answer comprehension questions is a central, yet unsolved goal of NLP. A key factor impeding its solution by machine learned systems is the limited availability of human-annotated data. Hermann et al. (2015) seek to solve this problem by creating over a million training examples by pairing CAW and Daily Mail news articles with their summarized bullet points, and show that a neural network can then be trained to give good performance on this task. In this paper, we conduct a thorough examination of this new reading comprehension task. Our primary aim is to understand what depth of language understanding is required to do well on this task. We approach this from one side by doing a careful hand-analysis of a small subset of the problems and from the other by showing that simple, carefully designed systems can obtain accuracies of 72.4% and 75.8% on these two datasets, exceeding current state-of-the-art results by over 5% and approaching what we believe is the ceiling for performance on this task1.

Original languageEnglish (US)
Title of host publication54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Long Papers
PublisherAssociation for Computational Linguistics (ACL)
Pages2358-2367
Number of pages10
ISBN (Electronic)9781510827585
DOIs
StatePublished - Jan 1 2016
Externally publishedYes
Event54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Berlin, Germany
Duration: Aug 7 2016Aug 12 2016

Publication series

Name54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Long Papers
Volume4

Other

Other54th Annual Meeting of the Association for Computational Linguistics, ACL 2016
CountryGermany
CityBerlin
Period8/7/168/12/16

All Science Journal Classification (ASJC) codes

  • Language and Linguistics
  • Linguistics and Language

Fingerprint Dive into the research topics of 'A thorough examination of the CNN/daily mail reading comprehension task'. Together they form a unique fingerprint.

  • Cite this

    Chen, D., Bolton, J., & Manning, C. D. (2016). A thorough examination of the CNN/daily mail reading comprehension task. In 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Long Papers (pp. 2358-2367). (54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Long Papers; Vol. 4). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/p16-1223