Evolution of reinforcement learning in uncertain environments: Emergence of risk-aversion and matching

Yael Niv, Daphna Joel, Isaac Meilijson, Eytan Ruppin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

Reinforcement learning (RL) is a fundamental process by which organisms learn to achieve a goal from interactions with the environment. Using Artificial Life techniques we derive (near-)optimal neuronal learning rules in a simple neural network model of decision-making in simulated bumblebees foraging for nectar. The resulting networks exhibit efficient RL, allowing the bees to respond rapidly to changes in reward contingencies. The evolved synaptic plasticity dynamics give rise to varying exploration/exploitation levels from which emerge the welldocumented foraging strategies of risk aversion and probability matching. These are shown to be a direct result of optimal RL, providing a biologically founded, parsimonious and novel explanation for these behaviors. Our results are corroborated by a rigorous mathematical analysis and by experiments in mobile robots.

Original languageEnglish (US)
Title of host publicationAdvances in Artificial Life - 6th European Conference, ECAL 2001, Proceedings
EditorsJozef Kelemen, Petr Sosik
PublisherSpringer Verlag
Pages252-261
Number of pages10
ISBN (Print)3540425675, 9783540425670
DOIs
StatePublished - 2001
Externally publishedYes
Event6th European Conference on Advances in Artificial Life, ECAL 2001 - Prague, Czech Republic
Duration: Sep 10 2001Sep 14 2001

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume2159
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other6th European Conference on Advances in Artificial Life, ECAL 2001
CountryCzech Republic
CityPrague
Period9/10/019/14/01

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Evolution of reinforcement learning in uncertain environments: Emergence of risk-aversion and matching'. Together they form a unique fingerprint.

  • Cite this

    Niv, Y., Joel, D., Meilijson, I., & Ruppin, E. (2001). Evolution of reinforcement learning in uncertain environments: Emergence of risk-aversion and matching. In J. Kelemen, & P. Sosik (Eds.), Advances in Artificial Life - 6th European Conference, ECAL 2001, Proceedings (pp. 252-261). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2159). Springer Verlag. https://doi.org/10.1007/3-540-44811-x_27