Parallel backward propagation of Morgan et.al.

The first method is described by Morgan et.al. [7]. Here the matrix products are calculated in parallel, while the subsums are sent around the processor cycle. At the end each processor has found its $\delta \text{'}$ .

$\delta$_k^':=(∑_jw_jk.δ_j) .o_k.(1-o_k)

The advantage of this parallelization is that the weight matrices are stored and modified only once, distributed on all processors. Therefor this parallelization has to calculate the partial sums of the error between each of the p-1 communication steps.