Your run failed at hour six and the internet will tell you what a learning rate is

When the loss goes to NaN

A decision tree for divergence. What to check first, what each observation rules out, and where gradient clipping actually belongs relative to the scaler.

serves the 45,870 view question on cost functions turning to NaN

Picking a learning rate and a schedule

The range test as it was actually described, why the chosen point is not the minimum of the curve, and how batch size moves it. Every starting value carries the conditions it was measured under.

serves the 28,313 view question on PyTorch schedulers

Reading metrics that disagree

Micro against macro against weighted, why F1 can sit below both precision and recall, and why ROC-AUC stays high while everything else collapses on imbalanced data. Worked by hand so you can check the arithmetic.

serves the 69,461 view question on F1 averaging

Choosing the optimizer

Where L2 and decoupled weight decay actually diverge, what each optimizer costs in bytes per parameter, and what the published comparisons do not establish.

serves the 19,208 view question on SGD against Adam

GPU memory for training and inference

The four terms, counted. Which ones move with batch size and which do not, and the three that are routinely misremembered.

Reading PyTorch shape errors

Real error strings, the actual shapes behind them, and the fix. The longest guide in the set.

The preflight checklist

What to verify before you commit hours of compute, each item with the cost of skipping it. Includes the single batch overfit test, which is the fastest correctness test there is.

Four scripts and two calculators

A memory budget calculator, a learning rate range test, a threshold sweep, and the overfit runner. Plus the KV cache and model memory calculators, offline, no network.

Python 3.9 or newer, standard library for the core paths, torch only where it cannot be avoided