TY - GEN
T1 - New Definitions and Evaluations for Saliency Methods
T2 - 36th Conference on Neural Information Processing Systems, NeurIPS 2022
AU - Gupta, Arushi
AU - Saunshi, Nikunj
AU - Yu, Dingli
AU - Lyu, Kaifeng
AU - Arora, Sanjeev
N1 - Funding Information:
Acknowledgements. We thank Ruth Fong for feedback on an earlier draft of this paper. We are also grateful to the valuable comments from various anonymous reviewers that helped improve the paper. This work is supported by funding from NSF, ONR, Simons Foundation, DARPA and SRC.
Publisher Copyright:
© 2022 Neural information processing systems foundation. All rights reserved.
PY - 2022
Y1 - 2022
N2 - Saliency methods compute heat maps that highlight portions of an input that were most important for the label assigned to it by a deep net. Evaluations of saliency methods convert this heat map into a new masked input by retaining the k highest-ranked pixels of the original input and replacing the rest with “uninformative” pixels, and checking if the net's output is mostly unchanged. This is usually seen as an explanation of the output, but the current paper highlights reasons why this inference of causality may be suspect. Inspired by logic concepts of completeness & soundness, it observes that the above type of evaluation focuses on completeness of the explanation, but ignores soundness. New evaluation metrics are introduced to capture both notions, while staying in an intrinsic framework-i.e., using the dataset and the net, but no separately trained nets, human evaluations, etc. A simple saliency method is described that matches or outperforms prior methods in the evaluations. Experiments also suggest new intrinsic justifications, based on soundness, for popular heuristic tricks such as TV regularization and upsampling.
AB - Saliency methods compute heat maps that highlight portions of an input that were most important for the label assigned to it by a deep net. Evaluations of saliency methods convert this heat map into a new masked input by retaining the k highest-ranked pixels of the original input and replacing the rest with “uninformative” pixels, and checking if the net's output is mostly unchanged. This is usually seen as an explanation of the output, but the current paper highlights reasons why this inference of causality may be suspect. Inspired by logic concepts of completeness & soundness, it observes that the above type of evaluation focuses on completeness of the explanation, but ignores soundness. New evaluation metrics are introduced to capture both notions, while staying in an intrinsic framework-i.e., using the dataset and the net, but no separately trained nets, human evaluations, etc. A simple saliency method is described that matches or outperforms prior methods in the evaluations. Experiments also suggest new intrinsic justifications, based on soundness, for popular heuristic tricks such as TV regularization and upsampling.
UR - http://www.scopus.com/inward/record.url?scp=85163209735&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85163209735&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85163209735
T3 - Advances in Neural Information Processing Systems
BT - Advances in Neural Information Processing Systems 35 - 36th Conference on Neural Information Processing Systems, NeurIPS 2022
A2 - Koyejo, S.
A2 - Mohamed, S.
A2 - Agarwal, A.
A2 - Belgrave, D.
A2 - Cho, K.
A2 - Oh, A.
PB - Neural information processing systems foundation
Y2 - 28 November 2022 through 9 December 2022
ER -