For the backward phase (figure 7) the neuron j in the output layer calculates the error between its
actual output value , known from the forward phase, and the expected nominal target value
:
The error is propagated backwards to the previous hidden layer.
The neuron i in a hidden layer calculates an error that is propagated
backwards again to its previous layer. Therefor a column of the weight matrix is used.
To minimize the error the weights of the projective edges of neuron i and the bias values in the receptive layer have to be changed. The old values have to be increased by:
is the training rate and has an empirical value:
.
The back-propagation algorithm optimizes the error by the method of gradient descent,
where ist the length of each step.