Speaker naming in movies

Mahmoud Azab, Mingzhe Wang, Max Smith, Noriyuki Kojima, Jia Deng, Rada Mihalcea

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

We propose a new model for speaker naming in movies that leverages visual, textual, and acoustic modalities in an unified optimization framework. To evaluate the performance of our model, we introduce a new dataset consisting of six episodes of the Big Bang Theory TV show and eighteen full movies covering different genres. Our experiments show that our multimodal model significantly outperforms several competitive baselines on the average weighted F-score metric. To demonstrate the effectiveness of our framework, we design an end-To-end memory network model that leverages our speaker naming model and achieves state-of-The-Art results on the subtitles task of the MovieQA 2017 Challenge.

Original languageEnglish (US)
Title of host publicationLong Papers
PublisherAssociation for Computational Linguistics (ACL)
Pages2206-2216
Number of pages11
ISBN (Electronic)9781948087278
StatePublished - Jan 1 2018
Externally publishedYes
Event2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018 - New Orleans, United States
Duration: Jun 1 2018Jun 6 2018

Publication series

NameNAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference
Volume1

Conference

Conference2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018
CountryUnited States
CityNew Orleans
Period6/1/186/6/18

All Science Journal Classification (ASJC) codes

  • Linguistics and Language
  • Language and Linguistics
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Speaker naming in movies'. Together they form a unique fingerprint.

  • Cite this

    Azab, M., Wang, M., Smith, M., Kojima, N., Deng, J., & Mihalcea, R. (2018). Speaker naming in movies. In Long Papers (pp. 2206-2216). (NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference; Vol. 1). Association for Computational Linguistics (ACL).