Incremental Training Based on Input Space Partitioning and Ordered Attribute Presentation with Knock-Out

Abstract:

A neural network training method ID-BT (Incremental Discriminatory Batch Training) is presented in this paper. It separates the input space into two batches: significant and insignificant attributes, and order the attributes within each batch according to their individual discrimination ability before introducing them into the network. By ‘knocking-out’ insignificant attributes that are futile, the generalization accuracy of network training is increased. To improve ID-BT further, we also propose ID-BIT (Incremental Discriminatory Batch and Individual Training) which introduces the significant attributes individually and insignificant attributes as a batch. The architecture used for both methods employs some incremental learning algorithms. We tested our algorithm extensively using several widely used benchmark problems, i.e. PROBEN1. The simulation results show that these two methods outperform incremental training with an increasing input dimension (ITID) or conventional batch training; we are able to achieve better network performance in terms of generalization accuracy yet not compromising training time and network complexity.