Using Machine Learning to Infer Reasoning Provenance from User Interaction Log Data

Neesha Kodagoda, Sheila Pontis, Donal Simmie, Simon Attfield, B. L.William Wong, Ann Blandford, Chris Hankin

Research output: Contribution to journalArticle

8 Scopus citations

Abstract

The reconstruction of analysts' reasoning processes (reasoning provenance) during complex sensemaking tasks can support reflection and decision making. One potential approach to such reconstruction is to automatically infer reasoning from low-level user interaction logs. We explore a novel method for doing this using machine learning. Two user studies were conducted in which participants performed similar intelligence analysis tasks. In one study, participants used a standard web browser and word processor; in the other, they used a system called INVISQUE (Interactive Visual Search and Query Environment). Interaction logs were manually coded for cognitive actions based on captured think-aloud protocol and posttask interviews based on Klein, Phillips, Rall, and Pelusos's data/frame model of sensemaking as a conceptual framework. This analysis was then used to train an interaction frame mapper, which employed multiple machine learning models to learn relationships between the interaction logs and the codings. Our results show that, for one study at least, classification accuracy was significantly better than chance and compared reasonably to a reported manual provenance reconstruction method. We discuss our results in terms of variations in feature sets from the two studies and what this means for the development of the method for provenance capture and the evaluation of sensemaking systems.

Original languageEnglish (US)
Pages (from-to)23-41
Number of pages19
JournalJournal of Cognitive Engineering and Decision Making
Volume11
Issue number1
DOIs
StatePublished - Mar 1 2017
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Human Factors and Ergonomics
  • Engineering (miscellaneous)
  • Applied Psychology
  • Computer Science Applications

Keywords

  • analytic provenance
  • data provenance
  • data/frame model
  • interaction frame mapper
  • machine learning
  • reasoning provenance
  • sensemaking

Fingerprint Dive into the research topics of 'Using Machine Learning to Infer Reasoning Provenance from User Interaction Log Data'. Together they form a unique fingerprint.

  • Cite this