Basis Lab

Answers · How models work

What does it mean to use stride 2 in a convolution?

In shortUsing stride 2 keeps the filter the same but moves its window two positions at a time, so it produces about half as many outputs in each direction and about a quarter of the work in two dimensions.

Using stride 2 means the filter's weights are exactly the same as with stride 1, but the window is evaluated at every second position along each axis. The convolution still passes over all the input values; it simply computes half as many responses in one dimension and a quarter as many in two.

nout=⌊nin+2p−ks⌋+1n_{\text{out}} = \left\lfloor \frac{n_{\text{in}} + 2p - k}{s} \right\rfloor + 1

The arithmetic

Take a row of nine values and a three-wide window with no padding. At stride 1 the window can start at positions 0 through 6, which is seven outputs: ⌊(9−3)/1⌋+1=7\lfloor(9-3)/1\rfloor+1=7. At stride 2 it starts only at 0, 2, 4 and 6, which is four outputs: ⌊(9−3)/2⌋+1=⌊6/2⌋+1=4\lfloor(9-3)/2\rfloor+1=\lfloor6/2\rfloor+1=4. The window at start 2 covers values 2, 3 and 4, and the window at start 4 covers 4, 5 and 6, so the same three weights are applied each time. The only change is that no response is computed for starts 1, 3 or 5.

FIG 1A three-wide window on a row of nine, at four strides
stepstridepositions usedoutput lengthcost relative to stride onewhat happened
110, 1, 2, 3, 4, 5, 671Every position starts a window.
220, 2, 4, 64about one halfEvery second position starts a window.
330, 3, 63about one thirdEvery third position starts a window.
440, 42about one quarterEvery fourth position starts a window.
4 steps
In one dimension the output length falls roughly by the stride; in two dimensions the number of outputs falls by the stride squared, so stride 2 does about a quarter of the arithmetic.

What stride does not do

The natural guess is that stride 2 changes the filter or deletes every other pixel before the convolution. Neither happens. The kernel keeps its shape and its weights, and the input row is not shortened before the operation. In the nine-value example, value 1 is skipped as a starting position, but it is still read by the window that starts at 0; value 3 is read by the window that starts at 2. What is skipped is the computation of a response at those starts. That distinction matters because a pattern centred exactly on an odd position produces no output at that position, even though its pixels took part in neighbouring windows.

In two dimensions the same count happens along both axes, so the number of outputs falls by the square of the stride. A 32 by 32 input with a three-wide window, padding of one and stride 1 gives ⌊(32+2−3)/1⌋+1=32\lfloor(32+2-3)/1\rfloor+1=32 outputs per axis, or 32×32=102432\times32=1024 positions. The same input with stride 2 gives ⌊(32+2−3)/2⌋+1=⌊31/2⌋+1=15+1=16\lfloor(32+2-3)/2\rfloor+1=\lfloor31/2\rfloor+1=15+1=16 outputs per axis, or 16×16=25616\times16=256 positions. The arithmetic therefore falls from 1024 window evaluations to 256, a quarter. Padding of one is what makes the halving exact here: without it the output would be ⌊(32−3)/2⌋+1=14+1=15\lfloor(32-3)/2\rfloor+1=14+1=15.

The floor in the formula is not decoration. If the stride does not divide the available span, the final partial step is not taken, and a strip at the far edge is read by no window at all. A 32-value row with a three-wide window, no padding and stride 3 gives ⌊(32−3)/3⌋+1=⌊29/3⌋+1=9+1=10\lfloor(32-3)/3\rfloor+1=\lfloor29/3\rfloor+1=9+1=10 outputs. The starts are 0, 3, 6, 9, 12, 15, 18, 21, 24 and 27; the last window covers 27, 28 and 29, so values 30 and 31 never appear in any window. Nothing in the operation warns about this. It is why two implementations that differ only in rounding can disagree by one position at an edge.

What it gives up

Stride 2 discards positions that are never recovered: with starts at 0, 2, 4 and 6, no response is ever computed for a pattern that would need a window starting at an odd position. It also weakens the shift property. At stride 1, shifting the input by one position shifts the output by one position. At stride 2, shifting the input by one position does not produce a shifted copy of the output; the exact statement survives only for shifts that are multiples of 2. In practice later layers tolerate this, and the saving is large enough that most designs accept the trade, but the lost odd starts are not reconstructed by the following convolution.

The saving is why stride 2 is common and why larger strides are rare. Going from stride 1 to 2 halves resolution and quarters two-dimensional arithmetic, which is a real decision. Going from 6 to 7 changes the output size by almost nothing while discarding most of what the layer computed. Stride and padding are usually chosen together so the arithmetic stays clean: a size-preserving three-wide window with stride 2 on an even input halves it exactly, keeping resolutions as powers of two and keeping the floor term from biting.

Worth keeping

  • Stride 2 changes where the window stops, not the weights inside it.
  • With a three-wide window on nine values, stride 1 gives seven outputs and stride 2 gives four, at start positions 0, 2, 4 and 6.
  • The positions that are skipped are not removed from the input; they are still read by neighbouring windows, but no response is computed starting there.

Where this goes next

This question is one step of How a Filter Reads a Picture, a course that works through its subject from the beginning. The page that teaches this part of it is Edges, and Skipping Positions, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that pageThe whole course

Other questions people ask about this

  1. What do you give up when you pool or downsample a feature map?You give up exact position: each block of the map becomes one number, so later layers see a coarser grid, gain reach, and lose the ability to say where a response came from.
  2. Why do CNNs use the same weights everywhere instead of learning a separate filter for each spot?They share one small filter across every position because a pattern such as an edge means the same thing wherever it appears; the huge cut in parameters is a consequence, not the main reason.
  3. Why does a convolution make the image smaller if you don't pad it?A convolution shrinks the image because the window must fit completely inside it, so the first and last few positions have no full window. Padding adds extra values at the border so those positions can be used.
  4. What does a convolution actually calculate at a single point in the image?It multiplies each filter weight by the pixel beneath it, adds those products, and writes the single total at the matching position in the output.
  5. Why does the model get stuck repeating the same sentence over and over?The model is not stuck in a memory loop: once it has written a phrase, that phrase is in its context and makes writing it again more likely, so a decoder that always picks the top word cannot leave.
  6. Why does diffusion add Gaussian noise instead of any random corruption?Diffusion adds Gaussian noise because Gaussian errors combine exactly: each step keeps the spread fixed and gives a closed-form noisy picture at any later step, which makes the reverse step learnable.

Read alongside