Batch Normalization
Step-by-step · mean · variance · normalize · scale & shift
Batch mean (μ)—
Batch variance (σ²)—
Normalized (x - μ)/√(σ²+ε)—
Scaled & shifted (γ·norm+β)—
Input values—
Inference mode — During training, running mean & variance are updated via exponential moving average.
At inference, the running mean and running variance replace batch statistics.
This ensures consistent normalization regardless of batch size.