Exploiting Attention to Reveal Shortcomings in Memory Models

Kaylee Burns; Aida Nematzadeh; Erin Grant; Alison Gopnik; Thomas L. Griffiths

Exploiting Attention to Reveal Shortcomings in Memory Models

Kaylee Burns, Aida Nematzadeh, Erin Grant, Alison Gopnik, Thomas L. Griffiths

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The decision making processes of deep networks are difficult to understand and while their accuracy often improves with increased architectural complexity, so too does their opacity. Practical use of machine learning models, especially for question and answering applications, demands a system that is interpretable. We analyze the attention of a memory network model to reconcile contradictory performance on a challenging question-answering dataset that is inspired by theory-of-mind experiments. We equate success on questions to task classification, which explains not only test-time failures but also how well the model generalizes to new training conditions.

Original language	English (US)
Title of host publication	EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP
Subtitle of host publication	Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop
Publisher	Association for Computational Linguistics (ACL)
Pages	378-380
Number of pages	3
ISBN (Electronic)	9781948087711
State	Published - 2018
Event	1st Workshop on BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, co-located with the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018 - Brussels, Belgium Duration: Nov 1 2018 → …

Publication series

Name	EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop

Conference

Conference	1st Workshop on BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, co-located with the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018
Country/Territory	Belgium
City	Brussels
Period	11/1/18 → …

All Science Journal Classification (ASJC) codes

Computer Science Applications
Computational Theory and Mathematics
Information Systems

Cite this

Burns, K., Nematzadeh, A., Grant, E., Gopnik, A., & Griffiths, T. L. (2018). Exploiting Attention to Reveal Shortcomings in Memory Models. In EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop (pp. 378-380). (EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop). Association for Computational Linguistics (ACL).

Burns, Kaylee ; Nematzadeh, Aida ; Grant, Erin et al. / Exploiting Attention to Reveal Shortcomings in Memory Models. EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop. Association for Computational Linguistics (ACL), 2018. pp. 378-380 (EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop).

@inproceedings{1964695c200e4c3b8497e8e76e833caf,

title = "Exploiting Attention to Reveal Shortcomings in Memory Models",

abstract = "The decision making processes of deep networks are difficult to understand and while their accuracy often improves with increased architectural complexity, so too does their opacity. Practical use of machine learning models, especially for question and answering applications, demands a system that is interpretable. We analyze the attention of a memory network model to reconcile contradictory performance on a challenging question-answering dataset that is inspired by theory-of-mind experiments. We equate success on questions to task classification, which explains not only test-time failures but also how well the model generalizes to new training conditions.",

author = "Kaylee Burns and Aida Nematzadeh and Erin Grant and Alison Gopnik and Griffiths, {Thomas L.}",

note = "Publisher Copyright: {\textcopyright} 2018 Association for Computational Linguistics; 1st Workshop on BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, co-located with the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018 ; Conference date: 01-11-2018",

year = "2018",

language = "English (US)",

series = "EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop",

publisher = "Association for Computational Linguistics (ACL)",

pages = "378--380",

booktitle = "EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP",

}

Burns, K, Nematzadeh, A, Grant, E, Gopnik, A & Griffiths, TL 2018, Exploiting Attention to Reveal Shortcomings in Memory Models. in EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop. EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop, Association for Computational Linguistics (ACL), pp. 378-380, 1st Workshop on BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, co-located with the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018, Brussels, Belgium, 11/1/18.

Exploiting Attention to Reveal Shortcomings in Memory Models. / Burns, Kaylee; Nematzadeh, Aida; Grant, Erin et al.
EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop. Association for Computational Linguistics (ACL), 2018. p. 378-380 (EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Exploiting Attention to Reveal Shortcomings in Memory Models

AU - Burns, Kaylee

AU - Nematzadeh, Aida

AU - Grant, Erin

AU - Gopnik, Alison

AU - Griffiths, Thomas L.

PY - 2018

Y1 - 2018

N2 - The decision making processes of deep networks are difficult to understand and while their accuracy often improves with increased architectural complexity, so too does their opacity. Practical use of machine learning models, especially for question and answering applications, demands a system that is interpretable. We analyze the attention of a memory network model to reconcile contradictory performance on a challenging question-answering dataset that is inspired by theory-of-mind experiments. We equate success on questions to task classification, which explains not only test-time failures but also how well the model generalizes to new training conditions.

AB - The decision making processes of deep networks are difficult to understand and while their accuracy often improves with increased architectural complexity, so too does their opacity. Practical use of machine learning models, especially for question and answering applications, demands a system that is interpretable. We analyze the attention of a memory network model to reconcile contradictory performance on a challenging question-answering dataset that is inspired by theory-of-mind experiments. We equate success on questions to task classification, which explains not only test-time failures but also how well the model generalizes to new training conditions.

UR - http://www.scopus.com/inward/record.url?scp=85070342297&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85070342297&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85070342297

T3 - EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop

SP - 378

EP - 380

BT - EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP

PB - Association for Computational Linguistics (ACL)

T2 - 1st Workshop on BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, co-located with the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018

Y2 - 1 November 2018

ER -

Burns K, Nematzadeh A, Grant E, Gopnik A, Griffiths TL. Exploiting Attention to Reveal Shortcomings in Memory Models. In EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop. Association for Computational Linguistics (ACL). 2018. p. 378-380. (EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop).

Exploiting Attention to Reveal Shortcomings in Memory Models

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this