I’ve looked all over the internet for explanations of what exactly back propagation is and everyone either uses complicated mathematical language or complex codes to try to explain what back propagation. If someone who doesn’t know either wants to know what it is then how will they really grasp what it is?In this post, I would like to unveil the secrets of the universe with everyone and hopefully I’ll do a good job at it.According to Wikipedia, Backpropagation is a method used in artificial neural networks to calculate a gradient that is needed in the calculation of the weights to be used in the network.Backpropagation is commonly used by the gradient descent optimization algorithm to adjust the weight of neurons by calculating the gradient of the loss functionIf you have taken a basic elementary algebra class, you may have heard of the idea of a slope. Some people might think the idea of a slope is very insignificant but it is actually the game-changing concept that caused all the technological advancement within the last century.To know the slope of something means that you know the rate of something changing over a period of time. Knowing this gives us power to manipulate things to our advantage.Now you can think of a gradient as the slope of something in a higher dimension. I won’t go into details but that is the general gist of what a gradient is.Weights are the values that we want to use to adjust the outputs of our functions in each neuron. So say we have an output of 2 and we want to change the 2 into a 1, then we would multiply the 2 by a .5 to get the desired result. This means that .5 will be the weight in this case. In a way we are weighing down the output to what we want it to be.A neuron is simply just a function. A Neural Network(a bunch of neurons) is simply a bunch of functions. Each neuron also has an activation function that spits out a value for the next neuron to calculate.Think of these functions as how much of a yes or a no an input is. An example would be picture recognizing. When you feed the neural network a picture, the node will spit out a number between 0 and 1. Where 0 is being very NO and 1 being very YES. This process continues between every node until the very end. Which ever node has the highest number, between 0 and 1, would be the decision the machine makes.A loss function is just some function that we use to determine how correct the predicted output is from the real output. For example, we input a picture of a cat into the machine but the machine predicts that it’s a dinosaur. Clearly the machine is not doing a very good job. So we need some way to know how correct the machine is compared to the real data. Which is where the loss function comes in.Now that we have all the necessary understandings, we can go into the real sauce.Now what I am about to explain to you is going to either confuse the crap out of you or make you feel enlightened.Let’s pretend you are trying to build a door lock opening mechanism. This mechanism involves you pressing a button, which triggers a ball rolling down a platform and knocks over a switch that unlocks the door.Now lets think about this. There are a few components that we have to keep in mind. The 1st component being you pressing the button, the 2nd component is the ball rolling down a platform, and the 3rd component being the switch being knocked over.There is actually a lot of physics going on around here but let’s just focus on the ball rolling down the platform.Now when you create this mechanism, you want the door to ideally open in 3 seconds. But you don’t have any tools to measure the time and length, so all you can do is to create a platform through intuition.You build your first platform and let the ball roll and realized that it took 9 seconds for the door to open after pressing the button.So you go back to the platform and make the platform steeper.You performed the same trial and error over and over again until you got the ideal opening time.This my friend, is Backpropagation.Well true. But the idea is basically the same. In a Neural Net, we have weights assigned to each neuron. These weights will get multiplied by a certain input and modified through some activation function. The result of these activation function might not always be what we want.What backpropagation would do is that it will do some calculus (will be covered in another post) to determine the direction of increase/decrease, aka the gradient,(cut less of the platform or cut more of the platform) to achieve the best weights (ideal time the door opens). It then updates these weights every time it has created new weights and runs the neural net again(every trial you cut a piece of the platform to test).Eventually we will achieve the best possible weights that satisfies our desired accuracy.In my next post, I will discuss more in depth about the math that is involved with backpropagation.References and LinksFrom a quick cheer to a standing ovation, clap to show how much you enjoyed this story.Getting stuck 24/7