Nova Southeastern University,
FL 33315,
USA
International Business Machines,
NC 27709,
USA
Abstract
We present the results of computer simulations of several aspects of parallel programming in relation to massively parallel neural network algorithms. Backpropagation, as the most widely used massively parallel neural network learning algorithm, was used to investigate the different aspects of parallel programming with the intent of providing the fastest overall execution of the algorithm in both multiprocessor and multicomputer environments. Our paper discusses the results of an investigation into the suitability og backpropagation algorithm for currency, data parallelism and partitioning, synchronous iteration mechanisms, communication and sychronization ddelays, and performance on different parallel architectures.