Feature Selection for Modular Networks

Based on Incremental Training

Sheng-Uei Guan and Jun Liu

Department of Electrical and Computer Engineering, National University of Singapore

10 Kent Ridge Crescent, Singapore 119260

Abstract

Feature selection plays an important role in classification systems. Using classifier error rate as the evaluation function, feature selection is integrated with incremental training. A neural network classifier is implemented with an incremental training approach to detect and discard irrelevant features. By learning attributes one after another, our classifier can find out directly the attributes that make no contribution to classification. These attributes are marked and considered for removal. Incorporated with an FLD feature ranking scheme, three batch removal methods based on classifier error rate have been developed to discard irrelevant features. These feature selection methods reduce the computational complexity involved in searching among a large number of possible solutions significantly. Experimental results show that our feature selection method works well on several benchmark problems. The selected subsets are further validated by a Constructive Backpropagation (CBP) classifier, which confirms increased classification accuracy and reduced training cost.

Keywords

Feature selection, Classifier, Neural network, Feedforward neural network, Incremental training, Input attribute