Category Loss (Cost Functions)

The loss (cost function) generally calculates the difference between the target and predicted values. When training a model, the loss value is back propagated through the model allowing gradients to be calculated at each learnable layer. The gradients are later applied to the weights during the optimization step.