Loss Function
A loss function measures the discrepancy a training procedure tries to reduce. Losses often compare predictions with target outputs, with individual values combined into an objective for updating the model.
Also known as: cost function
Suppose a delivery-time model predicts 8 minutes when the target is 5. Squared error gives (8 − 5)² = 9; absolute error gives |8 − 5| = 3. Squaring makes larger errors contribute disproportionately. These are illustrative calculations, not scores from a deployed system.
A classifier can use cross-entropy to penalize probabilities assigned to the wrong category. A language model can use a similar objective for target tokens. Losses may be averaged over examples or combined with a regularization penalty. A loss API can return per-example values before that aggregation; it need not always return one scalar immediately.
The choice matters because the optimizer follows the objective it receives. A falling training loss does not establish good task success if the targets miss important requirements or the data differs from deployment. Evaluate the behavior users need on held-out examples, alongside the training objective.
Sources
- Keras: Losses — Defines losses as training objectives and documents targets, prediction inputs and reduction choices.
Go deeper
- Google: Linear regression loss course
Compare absolute and squared error with worked regression examples.