Department Of Electronic Engineering,
City University Of Hong Kong,
83, Tat Chee Avenue,
Kowloon,
Hong Kong
E-mail: eejyli@cityu.edu.hk
E-mail: eetchow@cityu.edu.hk
Fax: (852) 2788 7791
Abstract
The function approximation capabilities of first-order neural networks have been investigated rigorously, but few related works about higher-order neural networks have been reported. In this paper, we proved that higher-order neural networks can approximate any continuous function on a compact set with an arbitrary degree of accuracy, provided that the activation function belongs to C and is non-polynomial. According to this theory, we know that partially connected higher-order neural networks can approximate any continuous functions as fully connected neural networks can do. Based on this argument, we designed a partially connected higher order network to compare with the first-order network when maintaining almost the same number of connections. The theory is validated by applying to sunspots series and simulation results show that the partially connected higher-order network exhibits better convergence rate and generalisation ability compared to the first-order network.