⚡ Gradient Clipping Calculator
Prevent exploding gradients · interactive analysis
Gradient values per parameter (comma‑separated)
Clipping method
Clip by value (min/max)
Clip by norm (max L2 norm)
Clip by global norm
Clipping threshold
Compute clipping
Reset defaults
Recommended thresholds:
RNN 1.0
Transformer 1.0
CNN 5.0
Original global norm
—
Clipped global norm
—
Scaling factor
—
Method
clip_by_norm
Original
Clipped
Enter gradients and click compute.
📘 When to use each method
Clip by value
– bound each element individually. Use when you want to avoid extreme single values; may distort direction.
Clip by norm
– scale whole gradient if L2 norm > threshold. Preserves direction. Default for RNN, Transformer.
Global norm
– treat all parameters as one vector. Use for multi‑layer / shared parameters.
🔹 RNN: 1.0 | Transformer: 1.0 | CNN: 5.0
🧩 PyTorch / TF snippets
# PyTorch (clip by norm)
torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
# TensorFlow (clip by global norm)
tf.clip_by_global_norm(gradients, clip_norm=1.0)
Snippet updates with selected method & threshold.