TY - JOUR
T1 - Moving beyond generalization to accurate interpretation of flexible models
AU - Genkin, Mikhail
AU - Engel, Tatiana A.
N1 - Funding Information:
This work was supported by NIH grant no. R01 EB026949 and the Swartz Foundation. We thank G. Angeris for help in the early stages of the project, K. Haas for useful discussions and P. Koo, J. Jansen, A. Siepel, J. Kinney and T. Janowitz for their thoughtful comments on the manuscript. We thank N.A. Steinmetz and T. Moore for sharing the electrophysiological data, which are presented in ref. 35 and are archived at the Stanford Neuroscience Institute server at Stanford University.
Publisher Copyright:
© 2020, The Author(s), under exclusive licence to Springer Nature Limited.
PY - 2020/11
Y1 - 2020/11
N2 - Machine learning optimizes flexible models to predict data. In scientific applications, there is a rising interest in interpreting these flexible models to derive hypotheses from data. However, it is unknown whether good data prediction guarantees the accurate interpretation of flexible models. Here, we test this connection using a flexible, yet intrinsically interpretable framework for modelling neural dynamics. We find that many models discovered during optimization predict data equally well, yet they fail to match the correct hypothesis. We develop an alternative approach that identifies models with correct interpretation by comparing model features across data samples to separate true features from noise. We illustrate our findings using recordings of spiking activity from the visual cortex of monkeys performing a fixation task. Our results reveal that good predictions cannot substitute for accurate interpretation of flexible models and offer a principled approach to identify models with correct interpretation.
AB - Machine learning optimizes flexible models to predict data. In scientific applications, there is a rising interest in interpreting these flexible models to derive hypotheses from data. However, it is unknown whether good data prediction guarantees the accurate interpretation of flexible models. Here, we test this connection using a flexible, yet intrinsically interpretable framework for modelling neural dynamics. We find that many models discovered during optimization predict data equally well, yet they fail to match the correct hypothesis. We develop an alternative approach that identifies models with correct interpretation by comparing model features across data samples to separate true features from noise. We illustrate our findings using recordings of spiking activity from the visual cortex of monkeys performing a fixation task. Our results reveal that good predictions cannot substitute for accurate interpretation of flexible models and offer a principled approach to identify models with correct interpretation.
UR - http://www.scopus.com/inward/record.url?scp=85093941325&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85093941325&partnerID=8YFLogxK
U2 - 10.1038/s42256-020-00242-6
DO - 10.1038/s42256-020-00242-6
M3 - Article
C2 - 36451696
AN - SCOPUS:85093941325
SN - 2522-5839
VL - 2
SP - 674
EP - 683
JO - Nature Machine Intelligence
JF - Nature Machine Intelligence
IS - 11
ER -