TUTOR: Training Neural Networks Using Decision Rules as Model Priors

Shayan Hassantabar, Prerit Terway, Niraj K. Jha

Research output: Contribution to journalArticlepeer-review

Abstract

The human brain has the ability to carry out new tasks with limited experience. It utilizes prior learning experiences to adapt the solution strategy to new domains. On the other hand, deep neural networks (DNNs) generally need large amounts of data and computational resources for training. However, this requirement is not met in many settings. To address these challenges, we propose the TUTOR DNN synthesis framework. TUTOR targets tabular datasets. It synthesizes accurate DNN models with limited available data and reduced memory/computational requirements. It consists of three sequential steps. The first step involves generation, verification, and labeling of synthetic data. The synthetic data generation module targets both categorical and continuous features. TUTOR generates synthetic data from the same probability distribution as the real data. It then verifies the integrity of the generated synthetic data using a semantic integrity classifier module. It labels synthetic data based on a set of rules extracted from the real dataset. Next, TUTOR uses two training schemes that combine synthetic and training data to learn the DNN model parameters. These two schemes focus on two different ways in which synthetic data can be used to derive a prior on the model parameters and, hence, provide a better DNN initialization for training with real data. In the third step, TUTOR employs a growand-prune synthesis paradigm to learn both the weights and the architecture of the DNN to reduce model size while ensuring its accuracy. We evaluate the performance of TUTOR on nine datasets of various sizes. We show that in comparison to fullyconnected DNNs, TUTOR, on an average, reduces the need for data by 5.9× (geometric mean), improves accuracy by 3.4%, and reduces the number of parameters (floating-point operations) by 4.7× (4.3×) (geometric mean). Thus, TUTOR enables less datahungry, more accurate, and more compact DNN synthesis.

Original languageEnglish (US)
Pages (from-to)1
Number of pages1
JournalIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
DOIs
StateAccepted/In press - 2022

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design
  • Electrical and Electronic Engineering

Keywords

  • Architecture synthesis
  • Artificial intelligence
  • Computational modeling
  • Computer architecture
  • Data models
  • Decision trees
  • Task analysis
  • Training
  • compact network
  • compression
  • deep neural network
  • synthetic data generation.

Fingerprint

Dive into the research topics of 'TUTOR: Training Neural Networks Using Decision Rules as Model Priors'. Together they form a unique fingerprint.

Cite this