References
Tags: concept
Sources:
Related notes:
Negative Log-Likelihood (NLL)
Cross-Entropy
Updates:
Notes {{word-count}}
Summary:
Key points:
The Loss Function quantifies how bad ΞΈ\thetaΞΈ is.
We want the least bad ΞΈ\thetaΞΈ.
Negative Log-Likelihood (NLL) is sometimes also called as Cross-Entropy.
Examples
ββilogβ‘pΞΈ(yiβ£xi)-\sum_{i} \log p_{\theta}\left(y_{i} \mid x_{i}\right)ββiβlogpΞΈβ(yiββ£xiβ)
The ββilogβ‘pΞΈ(yiβ£xi)-\sum_{i} \log p_{\theta}\left(y_{i} \mid x_{i}\right)ββiβlogpΞΈβ(yiββ£xiβ) part is also a Loss Function.
Zero-One Loss
ββiΞ΄(fΞΈ(xi)=yi)-\sum_{i} \delta\left(f_{\theta}\left(x_{i}\right)=y_{i}\right)ββiβΞ΄(fΞΈβ(xiβ)=yiβ)
Mean Squared Error
βi12β£fΞΈ(xi)βyiβ£2\sum_{i} \frac{1}{2}\left|f_{\theta}\left(x_{i}\right)-y_{i}\right|^{2}βiβ21ββ£fΞΈβ(xiβ)βyiββ£2
Mean Squared Error is actually Negative Log-Likelihood (NLL).
The optimizer searches the model class to find the model that minimizes the Loss Function.
Loss Function measures if one model in the model class is better than another.
Loss Function