Comparing the inductive biases of simple neural networks and Bayesian models

Thomas L. Griffiths, Joseph L. Austerweil, Vincent G. Berthiaume

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Understanding the relationship between connectionist and probabilistic models is important for evaluating the compatibility of these approaches. We use mathematical analyses and computer simulations to show that a linear neural network can approximate the generalization performance of a probabilistic model of property induction, and that training this network by gradient descent with early stopping results in similar performance to Bayesian inference with a particular prior. However, this prior differs from distributions defined using discrete structure, suggesting that neural networks have inductive biases that can be differentiated from probabilistic models with structured representations.

Original languageEnglish (US)
Title of host publicationBuilding Bridges Across Cognitive Sciences Around the World - Proceedings of the 34th Annual Meeting of the Cognitive Science Society, CogSci 2012
EditorsNaomi Miyake, David Peebles, Richard P. Cooper
PublisherThe Cognitive Science Society
Pages402-407
Number of pages6
ISBN (Electronic)9780976831884
StatePublished - 2012
Externally publishedYes
Event34th Annual Meeting of the Cognitive Science Society: Building Bridges Across Cognitive Sciences Around the World, CogSci 2012 - Sapporo, Japan
Duration: Aug 1 2012Aug 4 2012

Publication series

NameBuilding Bridges Across Cognitive Sciences Around the World - Proceedings of the 34th Annual Meeting of the Cognitive Science Society, CogSci 2012

Conference

Conference34th Annual Meeting of the Cognitive Science Society: Building Bridges Across Cognitive Sciences Around the World, CogSci 2012
Country/TerritoryJapan
CitySapporo
Period8/1/128/4/12

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Science Applications
  • Human-Computer Interaction
  • Cognitive Neuroscience

Keywords

  • Bayesian modeling
  • connectionism
  • inductive biases
  • property induction

Fingerprint

Dive into the research topics of 'Comparing the inductive biases of simple neural networks and Bayesian models'. Together they form a unique fingerprint.

Cite this