K.N. Gurney
Synopsis
A general model of so-called 'RAM-based nodes' which uses continuous sites values on the hypercube and processes analogue inputs is developed. A closed expression is given for the node's functionality and is shown to be equivalent to that for the sigma-pi nodes described by Rumelhart and McClleland (1986) of which, the customary semilinear units or TLUs are a subset. This is one of several points of contact developed between RAM-based nodes and those that use sums of weights of input terms. Points of contrast are also discussed in the context of generalisation, training and implementation. It is shown that the models may be instantiated with time stochastic processing and RAM implementations realised by quantising the site values.
Key words:
Neural Nets, Sigma-pi, Higher order, RAM nets, Hypercubes
1. Introduction
Networks of arbitrary Boolean functions (Aleksander, 1973; Aleksander & Stonham, 1979; Aleksander, Thomas & Bowden, 1984; Martland, 1987; Milligan, 1988; Austin, 1987) and, more recently Probabilistic Logic Nodes (Myers & Aleksander, 1988; Myers, 1989) and pRAMS (Gorse & Taylor, 1990) have traditionally been viewed as a radical alternative to the use of nodes whose activation is defined by a weighted sum of inputs. Part of the motivation for their use is that they readily lend themselves to implementation in RAM technology, leading to their collective title, 'RAM-based node' and their networks as 'RAM-nets'. This has placed them somewhat outside the mainstream of Neural Net research which has usually dealt with McCulloch and Pitts type neurons ( ILUs), or semilinear nodes which have a sigmoidal output function. Among the differences are that the latter can deal with analogue inputs and may have their functionality expressed in a closed analytic form, allowing proof of learning convergence in a supervised regime.
It is shown in this paper that the RAM-nets may be thought of as having both these properties. In particular, the classic training schemes using Reward Penalty (Barto & Jordan, 1987) and Back-propagation (Rumelhart & McClelland, 1986) may be applied. The route to this uses models with continuous (later quantised) values at each address and time stochastic processing of analogue inputs. There is a closed form for the functionality which may be expressed in a way equivalent to that for the sigma-pi units described by Rumelhart & McClelland (1986). These higher order nodes have, of course as a subset, the conventional semilinear units.
The models used are not bound to any hardware implementation but may still be built using RAM devices. The nodes are considered to be defined by a population of values at the vetices or sites of a hypercube and will be referred to as cube-based or cubic. The process of training set generalisation may be thought of as related to the clustering of similar site values with respect to the hypercube topology. One variant of this scheme is the Multi-Cube-Unit (MCU) which sums the activation from several cubes before producing output. This constrains the functionality and is the closest equivalent to the TLU or semilinear unit.