Average Error Across All Data, Cost Function

The cost function is a function that measures how much average error exists across the entire dataset when a machine learning model is learning.

It calculates the average of all data losses, thereby helping determine how well the model has been trained.

Similar to the loss function, the cost function uses various functions such as Mean Squared Error, Mean Absolute Error, Cross-Entropy, etc.

While the loss function and cost function are similar concepts, they differ in the unit of data they process.

The loss function refers to the error calculated for individual data sample.

For instance, when calculating the difference between the predicted value and the actual value for a single input data, the loss function is used.

On the other hand, the cost function represents the average error for the entire data samples.

In essence, the cost function is the average of the loss values from multiple data points.

The goal of a machine learning model during training is to minimize the cost function.

While the loss function helps track the model's performance on individual samples, the cost function is what gets optimized to ensure the model learns effectively from the entire training dataset.