Understanding exploration in humans and machines by formalizing the function of curiosity

Recent work in machine learning has demonstrated the benefits of providing artificial agents with a sense of curiosity — a form of intrinsic reward that supports exploration. Two strategies have emerged for defining these rewards: favoring novelty and pursuing prediction errors. Psychological theories of curiosity have also emphasized these two factors. We show how these two literatures can be connected by understanding the function of curiosity, which requires thinking about the abstract computational problem that both humans and machines face as they explore their world.