A Novel Multi-Stage Training Approach for Human Activity Recognition from Multimodal Wearable Sensor Data Using Deep Neural Network

Tanvir Mahmud, A. Q.M. Sazzad Sayyed, Shaikh Anowarul Fattah, Sun Yuan Kung

Research output: Contribution to journalArticlepeer-review

32 Scopus citations


Deep neural network is an effective choice to automatically recognize human actions utilizing data from various wearable sensors. These networks automate the process of feature extraction relying completely on data. However, various noises in time series data with complex inter-modal relationships among sensors make this process more complicated. In this article, we have proposed a novel multi-stage training approach that increases diversity in this feature extraction process to make accurate recognition of actions by combining varieties of features extracted from diverse perspectives. Initially, instead of using single type of transformation, numerous transformations are employed on time series data to obtain variegated representations of the features encoded in raw data. An efficient deep CNN architecture is proposed that can be individually trained to extract features from different transformed spaces. Later, these CNN feature extractors are merged into an optimal architecture finely tuned for optimizing diversified extracted features through a combined training stage or multiple sequential training stages. This approach offers the opportunity to explore the encoded features in raw sensor data utilizing multifarious observation windows with immense scope for efficient selection of features for final convergence. Extensive experimentations have been carried out in three publicly available datasets that provide outstanding performance consistently with average five-fold cross-validation accuracy of 99.29% on UCI HAR database, 99.02% on USC HAR database, and 97.21% on SKODA database outperforming other state-of-the-art approaches.

Original languageEnglish (US)
Article number9164933
Pages (from-to)1715-1726
Number of pages12
JournalIEEE Sensors Journal
Issue number2
StatePublished - Jan 15 2021

All Science Journal Classification (ASJC) codes

  • Instrumentation
  • Electrical and Electronic Engineering


  • CNN
  • Sensor data processing
  • activity recognition
  • feature learning
  • multi-stage training


Dive into the research topics of 'A Novel Multi-Stage Training Approach for Human Activity Recognition from Multimodal Wearable Sensor Data Using Deep Neural Network'. Together they form a unique fingerprint.

Cite this