Indoor human activity recognition using high-dimensional sensors and deep neural networks

Baptist Vandersmissen, Nicolas Knudde, Azarakhsh Jalalvand, Ivo Couckuyt, Tom Dhaene, Wesley De Neve

Research output: Contribution to journalArticlepeer-review

27 Scopus citations

Abstract

Many smart home applications rely on indoor human activity recognition. This challenge is currently primarily tackled by employing video camera sensors. However, the use of such sensors is characterized by fundamental technical deficiencies in an indoor environment, often also resulting in a breach of privacy. In contrast, a radar sensor resolves most of these flaws and maintains privacy in particular. In this paper, we investigate a novel approach toward automatic indoor human activity recognition, feeding high-dimensional radar and video camera sensor data into several deep neural networks. Furthermore, we explore the efficacy of sensor fusion to provide a solution in less than ideal circumstances. We validate our approach on two newly constructed and published data sets that consist of 2347 and 1505 samples distributed over six different types of gestures and events, respectively. From our analysis, we can conclude that, when considering a radar sensor, it is optimal to make use of a three-dimensional convolutional neural network that takes as input sequential range-Doppler maps. This model achieves 12.22% and 2.97% error rate on the gestures and the events data set, respectively. A pretrained residual network is employed to deal with the video camera sensor data and obtains 1.67% and 3.00% error rate on the same data sets. We show that there exists a clear benefit in combining both sensors to enable activity recognition in the case of less than ideal circumstances.

Original languageEnglish (US)
Pages (from-to)12295-12309
Number of pages15
JournalNeural Computing and Applications
Volume32
Issue number16
DOIs
StatePublished - Aug 1 2020
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Keywords

  • Activity recognition
  • Deep neural networks
  • High-dimensional sensors
  • Sensor fusion

Fingerprint

Dive into the research topics of 'Indoor human activity recognition using high-dimensional sensors and deep neural networks'. Together they form a unique fingerprint.

Cite this