TY - JOUR
T1 - Indoor human activity recognition using high-dimensional sensors and deep neural networks
AU - Vandersmissen, Baptist
AU - Knudde, Nicolas
AU - Jalalvand, Azarakhsh
AU - Couckuyt, Ivo
AU - Dhaene, Tom
AU - De Neve, Wesley
N1 - Publisher Copyright:
© 2019, Springer-Verlag London Ltd., part of Springer Nature.
PY - 2020/8/1
Y1 - 2020/8/1
N2 - Many smart home applications rely on indoor human activity recognition. This challenge is currently primarily tackled by employing video camera sensors. However, the use of such sensors is characterized by fundamental technical deficiencies in an indoor environment, often also resulting in a breach of privacy. In contrast, a radar sensor resolves most of these flaws and maintains privacy in particular. In this paper, we investigate a novel approach toward automatic indoor human activity recognition, feeding high-dimensional radar and video camera sensor data into several deep neural networks. Furthermore, we explore the efficacy of sensor fusion to provide a solution in less than ideal circumstances. We validate our approach on two newly constructed and published data sets that consist of 2347 and 1505 samples distributed over six different types of gestures and events, respectively. From our analysis, we can conclude that, when considering a radar sensor, it is optimal to make use of a three-dimensional convolutional neural network that takes as input sequential range-Doppler maps. This model achieves 12.22% and 2.97% error rate on the gestures and the events data set, respectively. A pretrained residual network is employed to deal with the video camera sensor data and obtains 1.67% and 3.00% error rate on the same data sets. We show that there exists a clear benefit in combining both sensors to enable activity recognition in the case of less than ideal circumstances.
AB - Many smart home applications rely on indoor human activity recognition. This challenge is currently primarily tackled by employing video camera sensors. However, the use of such sensors is characterized by fundamental technical deficiencies in an indoor environment, often also resulting in a breach of privacy. In contrast, a radar sensor resolves most of these flaws and maintains privacy in particular. In this paper, we investigate a novel approach toward automatic indoor human activity recognition, feeding high-dimensional radar and video camera sensor data into several deep neural networks. Furthermore, we explore the efficacy of sensor fusion to provide a solution in less than ideal circumstances. We validate our approach on two newly constructed and published data sets that consist of 2347 and 1505 samples distributed over six different types of gestures and events, respectively. From our analysis, we can conclude that, when considering a radar sensor, it is optimal to make use of a three-dimensional convolutional neural network that takes as input sequential range-Doppler maps. This model achieves 12.22% and 2.97% error rate on the gestures and the events data set, respectively. A pretrained residual network is employed to deal with the video camera sensor data and obtains 1.67% and 3.00% error rate on the same data sets. We show that there exists a clear benefit in combining both sensors to enable activity recognition in the case of less than ideal circumstances.
KW - Activity recognition
KW - Deep neural networks
KW - High-dimensional sensors
KW - Sensor fusion
UR - http://www.scopus.com/inward/record.url?scp=85070270718&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85070270718&partnerID=8YFLogxK
U2 - 10.1007/s00521-019-04408-1
DO - 10.1007/s00521-019-04408-1
M3 - Article
AN - SCOPUS:85070270718
SN - 0941-0643
VL - 32
SP - 12295
EP - 12309
JO - Neural Computing and Applications
JF - Neural Computing and Applications
IS - 16
ER -