Backward Propagation

For the backward phase (figure 7) the neuron j in the output layer calculates the error between its actual output value $o$_j , known from the forward phase, and the expected nominal target value $t$_j :

$\delta$_j:= (t_j- o_j) .f'(a_j)_{oj.(1 - oj)}

The error $\delta$_j is propagated backwards to the previous hidden layer.

The neuron i in a hidden layer calculates an error $\delta \text{'}$_i that is propagated backwards again to its previous layer. Therefor a column of the weight matrix is used.

$\delta \text{'}$_i:= (∑_jw_ji.δ_j) .f'(a_i)_{oi.(1 - oi)}

To minimize the error the weights of the projective edges of neuron i and the bias values in the receptive layer have to be changed. The old values have to be increased by:

$\Delta w$ji	$=$	$\eta .\delta$j.oi
j	$=$	$\eta .\delta$j.

$\eta$ is the training rate and has an empirical value: $\eta \approx 1$ .

The back-propagation algorithm optimizes the error by the method of gradient descent, where $\eta$ ist the length of each step.