Learning to soar in turbulent environments

Gautam Reddy, Antonio Celani, Terrence J. Sejnowski, Massimo Vergassola

Research output: Contribution to journalArticlepeer-review

115 Scopus citations

Abstract

Birds and gliders exploit warm, rising atmospheric currents (thermals) to reach heights comparable to low-lying clouds with a reduced expenditure of energy. This strategy of flight (thermal soaring) is frequently used by migratory birds. Soaring provides a remarkable instance of complex decision making in biology and requires a longterm strategy to effectively use the ascending thermals. Furthermore, the problem is technologically relevant to extend the flying range of autonomous gliders. Thermal soaring is commonly observed in the atmospheric convective boundary layer on warm, sunny days. The formation of thermals unavoidably generates strong turbulent fluctuations, which constitute an essential element of soaring. Here, we approach soaring flight as a problem of learning to navigate complex, highly fluctuating turbulent environments. We simulate the atmospheric boundary layer by numerical models of turbulent convective flow and combine them with model-free, experience-based, reinforcement learning algorithms to train the gliders. For the learned policies in the regimes of moderate and strong turbulence levels, the glider adopts an increasingly conservative policy as turbulence levels increase, quantifying the degree of risk affordable in turbulent environments. Reinforcement learning uncovers those sensorimotor cues that permit effective control over soaring in turbulent environments.

Original languageEnglish (US)
Pages (from-to)E4877-E4884
JournalProceedings of the National Academy of Sciences of the United States of America
Volume113
Issue number33
DOIs
StatePublished - Aug 16 2016
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • General

Keywords

  • Navigation
  • Reinforcement learning
  • Thermal soaring
  • Turbulence

Fingerprint

Dive into the research topics of 'Learning to soar in turbulent environments'. Together they form a unique fingerprint.

Cite this