Batch Normalization
Posted on Thu 16 February 2023 in Mathematics • Tagged with Mathematics, Machine Learning, Deep Learning
$$ \require{physics} \newcommand{B}{\mathcal{B}}$$
Batch Normalization was proposed by Ioffe and Szegedy in 2015, and it spawned several normalization techniques that are used in SOTA models today (layer norm, weight norm, etc). Batch normalization normalizes the output of each layer based on the mean and variance of the …
Continue reading