ml0x.com / tools / pytorch error explainer

PyTorch error explainer: paste the error, get the cause and the fix

For shape and size errors (mat1 and mat2 shapes cannot be multiplied, must match the size of tensor b, expected size, expected dim, size mismatch), dtype and device errors, and CUDA out of memory. It reads the numbers in your message and tells you which dimension is wrong. Everything runs in your browser.

Catch shape errors before training, not after

Most of the errors on this page surface minutes into a run, or on the last small batch. Tensor Preflight runs a traced forward pass locally, writes an HTML and JSON report of shapes, dtypes and devices per module, and keeps an output contract beside your code, so the same mistake fails fast next time.

See Tensor Preflight, $29 once

Python 3.9+ and PyTorch 2.8+. Tested on CPU and Apple MPS; CUDA is untested by its author.

Every error this explainer knows

RuntimeError: mat1 and mat2 shapes cannot be multiplied (AxB and CxD)

An nn.Linear (or torch.mm/@) got an input whose last dimension B is not the layer's in_features C. A is the number of rows (usually batch size), D is out_features. It almost always happens on the first Linear after a Conv or Flatten, when the flattened size differs from the number you typed into nn.Linear, often because the image size changed.

Fix

  • Set the layer to the size that actually arrives: nn.Linear(B, D).
  • Better: let PyTorch infer it with nn.LazyLinear(D), or compute it with one dummy forward pass (model.features(torch.zeros(1, 3, H, W)).numel()).
  • If A looks wrong too (for example A = batch x channels), you flattened the wrong dimensions. Use torch.flatten(x, 1) or nn.Flatten(), never x.view(-1, C) with a hard-coded C.
x = self.features(x)           # (N, 64, 7, 7)
x = torch.flatten(x, 1)         # (N, 3136)
self.fc = nn.Linear(64 * 7 * 7, 10)   # in_features must be 3136
# or: self.fc = nn.LazyLinear(10)

RuntimeError: The size of tensor a (X) must match the size of tensor b (Y) at non-singleton dimension Z

An element-wise operation (+, -, *, a loss, a mask) got two tensors whose sizes differ at dimension Z, and neither size is 1, so broadcasting cannot stretch one to the other. Common causes: predictions (N, 1) against targets (N), a sequence padded to a different length, an attention mask built for another sequence length, or the last batch being smaller than a tensor you cached.

Fix

  • Print both shapes right before the line: print(a.shape, b.shape).
  • If one side has an extra size-1 dimension, line them up with .squeeze(-1) or .unsqueeze(-1).
  • If the sizes are sequence lengths, pad or truncate both to the same length, and build masks from the actual batch.
  • If it fails only on the last batch, do not cache tensors sized to batch_size; use x.size(0), or set drop_last=True.
pred = model(x).squeeze(-1)   # (N, 1) -> (N)
loss = F.mse_loss(pred, y)     # y: (N)

RuntimeError: Sizes of tensors must match except in dimension D. Expected size X but got size Y for tensor number K in the list

torch.cat joins tensors along one dimension D, and every other dimension must be identical. Tensor number K has size Y where the first tensor has X. In U-Nets and skip connections this means the encoder and decoder feature maps differ by a pixel or two, because the input size is not divisible by 2 to the number of down-sampling steps. In sequence code it means different lengths.

Fix

  • Print the shapes of every tensor in the list.
  • U-Net / skip connection: resize or crop to match, e.g. F.interpolate(x, size=skip.shape[-2:]), or pad the input so H and W are divisible by 2depth.
  • Sequences: pad to the same length with torch.nn.utils.rnn.pad_sequence.
  • Check you are concatenating along the right dimension: features are usually dim=1 for (N, C, ...) and dim=-1 for (N, T, C).
x = self.up(x)
if x.shape[-2:] != skip.shape[-2:]:
    x = F.interpolate(x, size=skip.shape[-2:], mode="nearest")
x = torch.cat([x, skip], dim=1)

RuntimeError: stack expects each tensor to be equal size, but got [..] at entry 0 and [..] at entry N

torch.stack needs every tensor to have exactly the same shape. You usually see this from a DataLoader: the default collate function stacks samples into a batch, and one sample has a different shape. Typical causes: images of different sizes, a grayscale image among RGB ones (1 vs 3 channels), or variable-length text.

Fix

  • Images: add transforms.Resize((H, W)) (or a crop) to every sample, and convert every image with .convert("RGB").
  • Variable-length sequences: write a collate_fn that pads (pad_sequence) and returns the lengths.
  • Find the bad sample: loop over the dataset and print any index whose shape differs from item 0.
img = Image.open(path).convert("RGB")     # never 1 or 4 channels
tfm = transforms.Compose([transforms.Resize((224, 224)), transforms.ToTensor()])

RuntimeError: Given groups=1, weight of size [O, I, k, k], expected input[N, C, H, W] to have I channels, but got C channels instead

The convolution was built with in_channels=I, but the input has C channels in dimension 1. Two usual causes: the data has a different channel count (grayscale 1 vs RGB 3, RGBA 4), or the tensor is channels-last (N, H, W, C) as it comes from NumPy, PIL or OpenCV, so PyTorch reads the height or width as channels.

Fix

  • Channels-last input: x = x.permute(0, 3, 1, 2) (or use transforms.ToTensor(), which does it for you).
  • Grayscale into an RGB model: x.repeat(1, 3, 1, 1), or change the first layer to in_channels=1.
  • RGBA images: .convert("RGB") when loading.
  • If this is not the first conv, the previous layer's out_channels does not equal this layer's in_channels.
x = torch.from_numpy(batch)        # (N, H, W, 3) from NumPy/OpenCV
x = x.permute(0, 3, 1, 2).float()  # (N, 3, H, W)

RuntimeError: Expected 3D (unbatched) or 4D (batched) input to conv2d, but got input of size: [...]

Conv2d wants (N, C, H, W) or (C, H, W). You passed a tensor with the wrong number of dimensions: a single (H, W) image, a batch that lost its channel dimension (N, H, W), or an extra dimension such as (N, 1, C, H, W) from stacking a list of batches. Conv1d has the same rule with 2D/3D, Conv3d with 4D/5D.

Fix

  • Single grayscale image (H, W): x.unsqueeze(0).unsqueeze(0) gives (1, 1, H, W).
  • Batch without channel (N, H, W): x.unsqueeze(1).
  • Extra dimension of size 1: x.squeeze(1), or fix where you stack the batch.
x = img_tensor                 # (28, 28)
x = x.unsqueeze(0).unsqueeze(0) # (1, 1, 28, 28)
out = model(x)

ValueError: Expected input batch_size (X) to match target batch_size (Y)

The loss (usually CrossEntropyLoss or NLLLoss) got X rows of predictions and Y targets. When X is a multiple of Y, a view or reshape in the forward pass merged the batch with another dimension, typically x.view(-1, C) with a C that is smaller than the real flattened size. In sequence models it means logits were flattened to (N*T, V) but targets were not.

Fix

  • Replace x.view(-1, C) with x.view(x.size(0), -1) or torch.flatten(x, 1), and fix the next Linear's in_features.
  • Sequence models: flatten both sides the same way: loss(logits.view(-1, V), targets.view(-1)).
  • Check the DataLoader gives the same batch for inputs and labels (no extra unsqueeze).
x = torch.flatten(x, 1)            # keeps the batch dimension
logits = self.fc(x)                 # (N, num_classes)
loss = F.cross_entropy(logits, y)   # y: (N,)

RuntimeError: Expected target size [N, C], got [N, T]

CrossEntropyLoss treats dimension 1 of the input as the class dimension. Sequence logits from a Transformer or RNN are (N, T, C), so the loss reads T as the class count and expects a target of shape (N, C). The classes must be in dimension 1.

Fix

  • Move classes to dimension 1: F.cross_entropy(logits.permute(0, 2, 1), targets) with targets (N, T).
  • Or flatten both: F.cross_entropy(logits.reshape(-1, C), targets.reshape(-1)).
logits = model(tokens)               # (N, T, C)
loss = F.cross_entropy(logits.reshape(-1, logits.size(-1)), targets.reshape(-1))

RuntimeError: 0D or 1D target tensor expected, multi-target not supported

CrossEntropyLoss wants class indices of shape (N,) and dtype long. You gave it (N, 1) labels, or one-hot (N, C) labels.

Fix

  • Labels of shape (N, 1): y = y.squeeze(1).
  • One-hot labels: y = y.argmax(dim=1), or keep them as float probabilities (PyTorch 1.10+ accepts soft targets with the same shape as the input).
  • Make sure the dtype is torch.long: y = y.long().
loss = F.cross_entropy(logits, y.squeeze(1).long())

ValueError: Target size (torch.Size([N])) must be the same as input size (torch.Size([N, 1]))

BCEWithLogitsLoss and BCELoss need the target and the input to have the identical shape. A model with one output unit returns (N, 1) while labels are (N). The MSELoss version of this is a warning ("Using a target size that is different to the input size"), and it is worse: it silently broadcasts to (N, N) and trains on the wrong loss.

Fix

  • loss = criterion(logits.squeeze(-1), y.float()).
  • Or keep labels as (N, 1): y.float().unsqueeze(1).
  • Targets must be float for BCE, not long.
logits = model(x).squeeze(-1)                # (N,)
loss = F.binary_cross_entropy_with_logits(logits, y.float())

RuntimeError: shape '[...]' is invalid for input of size N

view or reshape must keep the total number of elements. The product of the shape you asked for does not divide or equal N. Usually a hard-coded size (batch size, image size, hidden size) is wrong, often only on the last, smaller batch.

Fix

  • Use -1 for exactly one dimension and the real sizes for the rest: x.view(x.size(0), -1).
  • Never hard-code the batch size in a reshape.
  • If N is not what you expected, print x.shape before the reshape; the bug is upstream.
x = x.view(x.size(0), -1)    # not x.view(32, 3136)

RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

view needs memory that is laid out contiguously. After transpose, permute or some slicing, the tensor is a strided view of the original data, and view cannot reinterpret it.

Fix

  • Use .reshape(...), which copies only when it must.
  • Or call .contiguous() first: x.transpose(1, 2).contiguous().view(N, T, -1) (the usual pattern in multi-head attention).
out = out.transpose(1, 2).contiguous().view(N, T, n_heads * head_dim)

RuntimeError: The expanded size of the tensor (X) must match the existing size (Y) at non-singleton dimension Z

expand (also used inside broadcasting assignments and some indexing) can only stretch dimensions of size 1. Dimension Z has size Y, which is not 1, and you asked for X. A common trigger is assigning into a slice of a pre-allocated tensor of another shape, or an attention mask expanded to the wrong sequence length.

Fix

  • Print the shape before expand; add the missing size-1 dimension with unsqueeze in the right place.
  • If you assign into a slice, check the slice shape equals the value shape.
  • Masks: build them from the current sequence length, not a cached maximum.
mask = mask[:, None, None, :]           # (N, 1, 1, T)
scores = scores.masked_fill(mask == 0, float("-inf"))

RuntimeError: Error(s) in loading state_dict: size mismatch for fc.weight: copying a param with shape torch.Size([A, B]) from checkpoint, the shape in current model is torch.Size([C, B])

The checkpoint was trained with a different architecture or head size than the model you built. Most often: a pretrained model with 1000 ImageNet classes loaded into a model you built for your own number of classes, a different vocabulary size, or a different hidden size.

Fix

  • Fine-tuning a new head: load the checkpoint first into the original architecture, then replace the head: model.fc = nn.Linear(model.fc.in_features, num_classes).
  • Or drop the mismatched keys before loading: filter the state dict to keys whose shapes match, then load_state_dict(sd, strict=False), and check the returned missing keys.
  • If every layer mismatches, you built the wrong model size (for example base vs large). Read the config saved with the checkpoint.
sd = torch.load(path, map_location="cpu")
own = model.state_dict()
sd = {k: v for k, v in sd.items() if k in own and v.shape == own[k].shape}
print(model.load_state_dict(sd, strict=False))

IndexError: Dimension out of range (expected to be in range of [-N, N-1], but got D)

You asked for dimension D of a tensor that has only N dimensions. Usually the tensor has fewer dimensions than the code assumes: a single sample without a batch dimension, a loss value that is already a scalar, or softmax(dim=1) on a 1D tensor.

Fix

  • Print x.dim() and x.shape at that line.
  • Missing batch dimension: x.unsqueeze(0) before the model.
  • Prefer negative dimensions for feature axes: softmax(dim=-1) works for both (C) and (N, C).
probs = logits.softmax(dim=-1)

RuntimeError: input.size(-1) must be equal to input_size. Expected X, got Y

An nn.RNN, nn.LSTM or nn.GRU was built with input_size=X, but the last dimension of its input is Y. Either the feature size changed (embedding dimension, number of features), or the input is laid out as (N, F, T) instead of (N, T, F).

Fix

  • If Y is your feature size, build the layer with input_size=Y.
  • If Y is your sequence length, the time and feature axes are swapped: x.transpose(1, 2).
  • With batch_first=True the input is (N, T, F); without it, (T, N, F).
self.lstm = nn.LSTM(input_size=emb_dim, hidden_size=256, batch_first=True)
out, _ = self.lstm(x)    # x: (N, T, emb_dim)

AssertionError: was expecting embedding dimension of X, but got Y

nn.MultiheadAttention or nn.TransformerEncoderLayer was built with embed_dim / d_model = X, and the last dimension of the query is Y. Two causes: the model width is different from the embedding output, or batch_first is wrong so the layer reads a different axis.

Fix

  • Make the embedding output size equal d_model, or add a projection nn.Linear(Y, X).
  • Check batch_first: with the default False the layer expects (T, N, E); pass batch_first=True for (N, T, E).
layer = nn.TransformerEncoderLayer(d_model=512, nhead=8, batch_first=True)
x = self.proj(x)   # (N, T, 512)

Expected dim X, got Y / got input tensor dim ... expected dim

A layer checked the size of a feature dimension and found Y where it was built for X. In attention, embedding and adapter code this is the model width (for example 512 vs 768); in vision models it is the channel or patch-embedding width.

Fix

  • Find the layer from the stack trace and print its configured size and the input shape.
  • If the input is right, rebuild the layer with the new size, or add a projection nn.Linear(Y, X).
  • If X and Y are a sequence length and a feature size, an axis is swapped: check batch_first and any transpose.
print(type(layer).__name__, x.shape)
proj = nn.Linear(in_dim, expected_dim)

RuntimeError: output with shape [1, H, W] doesn't match the broadcast shape [3, H, W]

An in-place operation (x.sub_(), x += ..., transforms.Normalize) cannot grow its output. The tensor has 1 channel and the other operand has 3. The classic case: Normalize(mean=[3 values], std=[3 values]) applied to a grayscale image.

Fix

  • Convert images to RGB when loading: Image.open(p).convert("RGB").
  • Or use single-channel normalisation for grayscale data: Normalize((0.5,), (0.5,)).
  • In your own code, use the out-of-place version: x = x - mean.
transforms.Compose([
    transforms.Grayscale(num_output_channels=3),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
])

Shape does not match expected size

A check comparing an actual shape with an expected one failed. The message comes from a library (a transform, a dataset, a model wrapper) or from an assertion. The two shapes in the message tell you which dimension differs and by how much.

Fix

  • Read which dimension differs. A factor of 2, 3 or 4 suggests channels or stride; a small difference suggests padding or rounding.
  • Print shapes before the call and compare with what the layer or library documents.
  • Paste the full trace into the box above: a more specific rule may match the line below the top one.
print(x.shape, expected_shape)

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

One operation mixes a tensor on the GPU with one on the CPU. Usually: the model is on CUDA but the batch is not, or a tensor created inside forward (a mask, positional ids, torch.zeros(...)) is on the CPU by default, or a plain tensor attribute was not registered as a buffer, so model.to(device) did not move it.

Fix

  • Move every batch: x, y = x.to(device), y.to(device).
  • Create tensors on the right device: torch.zeros(n, device=x.device), torch.arange(T, device=x.device).
  • Constant tensors in a module: self.register_buffer("pe", pe), so .to(device) moves them.
self.register_buffer("pos", torch.arange(max_len))
...
mask = torch.ones(T, T, device=x.device).tril()

RuntimeError: expected scalar type Float but found Double / mat1 and mat2 must have the same dtype

Two tensors in one operation have different dtypes. NumPy arrays become float64 (Double) when converted, while model weights are float32 (Float). Under mixed precision you can also get Half or BFloat16 against Float. Labels as float where a long is needed give the reverse.

Fix

  • Inputs from NumPy: torch.from_numpy(a).float(), or torch.tensor(a, dtype=torch.float32).
  • Mixed precision: run the forward pass inside torch.autocast(...) instead of calling .half() on some tensors.
  • Classification labels must be long; BCE targets must be float.
x = torch.from_numpy(features).float()
y = torch.from_numpy(labels).long()

RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

The input and the layer's weights differ in device (one says cuda, the other does not) or in dtype (Half vs Float). This is the older wording of the device and dtype errors above.

Fix

  • Device: move the input with x.to(next(model.parameters()).device).
  • Dtype: cast the input to the weight dtype, or use torch.autocast.
device = next(model.parameters()).device
x = x.to(device)

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate X GiB

The GPU does not have enough free memory for the next allocation. Training memory is weights + gradients + optimizer state (about 16 bytes per parameter with Adam in fp32) + activations, and activations grow with batch size x sequence length (or image size). If memory is fine for a while and then runs out, something keeps references across steps.

Fix

  • Halve the batch size; keep the effective batch with gradient accumulation.
  • Use mixed precision (torch.autocast with bf16 or fp16) and, for large models, activation checkpointing.
  • Evaluation: wrap it in torch.no_grad() (or torch.inference_mode()).
  • Leak check: accumulate loss.item(), not loss; do not keep outputs or hidden states from earlier steps.
  • If the message shows a lot of memory "reserved by PyTorch but unallocated", it is fragmentation: set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True.
  • Check other processes with nvidia-smi; a notebook kernel may still hold the GPU.
scaler_ctx = torch.autocast("cuda", dtype=torch.bfloat16)
for i, (x, y) in enumerate(loader):
    with scaler_ctx:
        loss = model(x).loss / accum
    loss.backward()
    if (i + 1) % accum == 0:
        opt.step(); opt.zero_grad(set_to_none=True)
    running += loss.item()

IndexError: index out of range in self

An nn.Embedding (or index_select/gather) got an index at or above its size. With text, a token id is greater than or equal to num_embeddings: the tokenizer and the embedding table disagree on the vocabulary size, or you added special tokens without resizing. Negative ids (like -100 padding) also trigger it.

Fix

  • Print ids.max(), ids.min() and embedding.num_embeddings.
  • After adding tokens to a Hugging Face tokenizer: model.resize_token_embeddings(len(tokenizer)).
  • Replace -100 or other padding values with the real pad id before the embedding; keep -100 only in the labels.
assert ids.max() < model.get_input_embeddings().num_embeddings, ids.max()

RuntimeError: CUDA error: device-side assert triggered

A check inside a GPU kernel failed. The real error is hidden because CUDA reports it late. In training code it is nearly always an index out of range: a class label that is greater than or equal to the number of classes, a label of -1, or a token id outside the embedding table.

Fix

  • Run the same batch on the CPU, which gives the real message, or set CUDA_LAUNCH_BLOCKING=1 to get the right line.
  • Check labels: y.min() >= 0 and y.max() < num_classes.
  • After this error the CUDA context is broken; restart the process (or the notebook kernel).
CUDA_LAUNCH_BLOCKING=1 python train.py
# or
assert 0 <= y.min() and y.max() < num_classes, (y.min(), y.max())

Questions

What does "mat1 and mat2 shapes cannot be multiplied" mean?

The input to an nn.Linear layer has a last dimension that differs from the layer's in_features. In (AxB and CxD), B is what arrives and C is what the layer expects. Set in_features to B, or use nn.LazyLinear.

How do I fix "Sizes of tensors must match except in dimension 1"?

torch.cat needs every dimension except the one you join along to be equal. In U-Nets, resize the upsampled feature map to the skip connection's height and width before concatenating.

How do I fix CUDA out of memory in PyTorch?

Reduce the batch size and use gradient accumulation, train in bf16 or fp16 with torch.autocast, wrap evaluation in torch.no_grad(), and accumulate loss.item() instead of loss. If a lot of memory is reserved but unallocated, set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True.

Is my error text sent anywhere?

No. The rules run in your browser. The page counts only which rule matched (for example mat1-mat2), never the text you paste.