Deep learning Back Propagation Gradient Descent and Chain Rule Derivation
Hi Iam vignesh and this is my first blog on Neural network on topic forward propagation and back propagation.This blog is for who want to visualise the backprogation,chain rule,gradient descent mathematically.There are many online content for backpropagation,chain rule and gradient descent explained, but here i try to explain the backpropagation,chain rule,gradient descent in mathematical approach.
Backpropagation, short for “backward propagation of errors,” is an algorithm for supervised learning of artificial neural networks using gradient descent. Given an artificial neural network and an error function, the method calculates the gradient of the error function with respect to the neural network’s weights. It is a generalization of the delta rule for perceptrons to multilayer feedforward neural networks.
Difference between back-propagation and forward-propagation?
In neural networks, you forward propagate to get the output and compare it with the real value to get the error.
Now, to minimize the error, you propagate backwards by finding the derivative of error with respect to each weight and then subtracting this value from the weight value.
Here I have shared the forward propagation, back propagation,chain rule and gradient descent derivation as pdf file.
Back Propagation pdf — link here
Gradient descent pdf — link here
Chain Rule pdf — link here
and also i shared the coding for gradient descent and back propagation
other link
