One Sentence Through GPT-2, Qwen3 and a Mamba Hybrid: Where Does the Name Come From?

One Sentence Through GPT-2, Qwen3 and a Mamba Hybrid: Where Does the Name Come From?#

We gave three small language models the same unfinished sentence:

In the morning, Alice gave Bob a book. Later, Bob gave the book back to

All three pick " Alice" as the next token: GPT-2 with probability 0.776, Qwen3-0.6B-Base with 0.980 and Granite-4.0-H-350M with 0.986. Granite is a hybrid: 28 of its 32 layers are Mamba2 layers and only 4 are attention layers. If we stop Granite after each layer and ask what it would say, " Alice" first comes out on top right after attention layer 27 (we count layers and positions from 0). So we switched off all four attention layers to see whether the name depends on them. It stayed at 0.988. Then we put about 1,400 tokens of unrelated text between Alice and the end of the question. Now the full model still answers 0.990, but without its attention layers it falls to 0.002.

To get there, we wrote each model's forward pass by hand, in a few dozen lines of PyTorch per model, and followed the last token, " to", from its ID to the final probabilities. This post walks that path one step at a time and shows the code for each step. The three models share the same frame and differ in a few cells, so the differences are easy to point at.

The three models#

GPT-2 is close to the original Transformer design. Qwen3 is a current dense model. Granite-4.0-H, from IBM, mixes Mamba2 and attention. All three are base models (no chat tuning), and we ran all of them on a CPU in float32.

Before trusting our own code, we compared its output with the Hugging Face transformers library on the same sentence. The largest difference in any logit (the raw score before probabilities) was 7.6e-5 for GPT-2, 2.9e-5 for Qwen3 and 4.2e-5 for Granite. So every number below comes from code that computes the same thing the library does.

To check that the models are not simply fond of the name Alice, we kept the sentence and changed only the name:

In the morning, Carol gave Bob a book. Later, Bob gave the book back to

We tried five names. Mirabel is on the list because it is rare: every tokenizer cuts it into two pieces, " Mir" and "abel", so for that row we score the first piece, " Mir". The table shows the probability of the name that was in the sentence.

In all 15 cases the name from the sentence is the top answer. The models are copying the name from earlier in the text, not reaching for a favorite.

Step 1: text becomes token IDs#

Each model has its own tokenizer, a fixed list that cuts text into pieces and gives each piece a number. Our sentence happens to become 18 pieces in all three. Other text does not split so evenly. The Korean greeting "안녕하세요" becomes 14 tokens in GPT-2, 3 in Qwen3 and 5 in Granite. GPT-2's list has few Korean entries, so it falls back to raw bytes.

The last piece is " to", with ID 284 in GPT-2 and 311 in the other two. We follow that one token from here on, because the next-token prediction is read from the last position.

Step 2: the ID picks a row of a table#

The embedding is a table with one row per vocabulary entry. Looking up a token means taking its row.

x = sd["transformer.wte.weight"][ids]           # GPT-2: (18, 768)
x = sd["model.embed_tokens.weight"][ids]        # Qwen3: (18, 1024)
x = sd["model.embed_tokens.weight"][ids] * 12   # Granite: (18, 768), scaled by 12

The row for " to" in Qwen3 starts with -0.0247, 0.0134, -0.0347, -0.0304 and continues for 1,024 numbers. Nothing in it says "this is the 18th token". The models handle word order in three different ways.

GPT-2 adds a second row for the position. It has another table with 1,024 rows, one per position, and adds row 17 to the token row:

x = sd["transformer.wte.weight"][ids] + sd["transformer.wpe.weight"][:T]

The position row is not small. For " to" at position 17, the position row has length 3.59 and the token row 2.52. The table also sets a hard limit: there is no row 1,024. So we could not run GPT-2 on the 1,410-token version of the sentence used later in this post.

Qwen3 rotates. It has no position table. Inside attention, the query and key vectors (introduced in Step 4) are turned by an angle that grows with position, a method called RoPE:

def rope(x, pos, theta):
    d = x.shape[-1]
    inv = 1.0 / theta ** (torch.arange(0, d, 2) / d)
    ang = pos[:, None].float() * inv[None]
    cos, sin = torch.cat([ang, ang], -1).cos(), torch.cat([ang, ang], -1).sin()
    x1, x2 = x[..., : d // 2], x[..., d // 2:]
    return x * cos + torch.cat([-x2, x1], -1) * sin

The point of the rotation is that the match score between two tokens depends only on how far apart they are, not on where they sit. To see this, we made one random query and one random key, each 128 numbers long (the size of one Qwen3 head). Any random pair shows the same pattern; the exact scores below are from our run. We rotated them as if they sat at two positions and took their dot product, which is the match score attention uses:

q, k = torch.randn(1, 1, 128), torch.randn(1, 1, 128)
def score(m, n):   # query at position m, key at position n
    return (rope(q, torch.tensor([m]), 1e6) * rope(k, torch.tensor([n]), 1e6)).sum().item()

Without any rotation the score is 11.796. At positions 17 and 5 it is 1.14666. At positions 112 and 100 the absolute positions are different but the gap is the same 12, and the score is again 1.14666. At 17 and 16, one apart, it is 10.646.

Granite does neither. Its config says position_embedding_type: nope. The attention layers add no position signal of their own. Order reaches the model through the Mamba2 layers, which read the tokens one after another, as Step 4 shows.

Step 3: rescale before each sub-step#

Each layer rescales a token's vector before using it. GPT-2 uses LayerNorm, which subtracts the mean and divides by the spread. Qwen3 and Granite use RMSNorm, which only divides by the root mean square:

def rms_norm(x, w, eps):
    return w * (x * torch.rsqrt(x.pow(2).mean(-1, keepdim=True) + eps))

For " to" entering the first layer of Qwen3, the root mean square goes from 0.0242 to 0.999, and the mean stays off zero at 0.034. GPT-2's LayerNorm takes its own " to" vector (token row plus position row) from 0.162 to 1.0 and sets the mean to exactly 0. Then both multiply by a learned weight w. RMSNorm skips the mean, which is a little less work, and most newer models use it.

Step 4: read from earlier tokens#

This is the only step where a token looks at other tokens, and it is where the three models differ most.

Attention (GPT-2, Qwen3 and four Granite layers). Every token makes three vectors from its own row: a query (what it is looking for), a key (what it offers) and a value (what it hands over). The last token compares its query with the keys of all 18 tokens, turns the scores into weights that sum to 1, and takes the weighted sum of values:

def attention(q, k, v, scale):
    rep = q.shape[0] // k.shape[0]                  # Qwen3: 16 // 8 = 2
    k, v = k.repeat_interleave(rep, 0), v.repeat_interleave(rep, 0)
    T = q.shape[1]
    scores = (q @ k.transpose(-1, -2)) * scale
    scores = scores.masked_fill(torch.triu(torch.ones(T, T, dtype=torch.bool), 1), float("-inf"))
    return scores.softmax(-1) @ v

The masked_fill line hides later tokens, so no token reads the future. GPT-2 runs 12 heads, each with its own keys and values. Qwen3 runs 16 query heads but only 8 key/value heads, and each key/value head is shared by two query heads (repeat_interleave). This is called grouped-query attention, and here it halves the keys and values the model has to keep in memory. Our attention roundup covers 28 variants of this one step.

Mamba2 (28 Granite layers). There is no comparison with every earlier token. The layer keeps a running state, a fixed-size table of numbers, and updates it once per token:

for t in range(T):
    decay = torch.exp(dt[t] * A)                  # A < 0, so 0 < decay < 1
    state = decay[:, None, None] * state + (dt[t][:, None] * xs[t])[:, :, None] * B[t][None, None, :]
    ys.append(state @ C[t] + D[:, None] * xs[t])

This is the core of the loop. The full layer also has a short convolution before it and a gate after it.

Each step shrinks the old state by decay and writes the current token in. dt is computed from the token itself, so a token can choose to write strongly or barely, and to keep or forget what came before. The output is read from the state with C. Because the loop runs in order, the token at position 5 has gone through 6 updates and the one at position 17 through 18. That is how Granite knows word order without a position table. The state has the same size at token 18 and at token 18,000. That is the main reason to use Mamba: memory does not grow with length.

Unrolling the loop gives a table of the same shape as attention's: how much each token takes from each earlier token. Below are both, for one attention layer and the Mamba2 layer right after it.

Both tables are empty above the diagonal. In the attention table most rows lean heavily on the first token. Models are known to park weight on the first token when nothing else is needed (the attention sink), so a large weight there does not mean "In" matters. A weight is also not a cause. To find out what a layer actually contributes, we switch it off later in this post.

Step 5: add back#

What the mixing step produces is added to the token's own vector, and the result moves on. The same happens after Step 6. This is the residual connection:

x = x + mix          # GPT-2, Qwen3
x = x + mix * 0.246  # Granite scales each update by its residual multiplier

Updates pile up layer after layer, so the vector mostly grows. For " to" after layer 0 and at its largest, the vector length goes from 52 to 512 in GPT-2, from 7 to 696 in Qwen3 and from 7 to 67 in Granite. The final norm in Step 7 brings it back to a fixed scale before the prediction.

Step 6: transform each token alone#

The FFN (feed-forward network) works on one token at a time. It widens the vector, applies a nonlinearity and narrows it again. GPT-2 uses GELU:

h = F.gelu(h @ W_in + b_in, approximate="tanh")   # 768 -> 3,072
x = x + h @ W_out + b_out                          # 3,072 -> 768

Qwen3 and Granite use SwiGLU, which computes two widened vectors and multiplies them, one passed through SiLU acting as a gate:

h = F.silu(h @ W_gate.T) * (h @ W_up.T)            # Qwen3: 1,024 -> 3,072
x = x + h @ W_down.T                               # 3,072 -> 1,024

Steps 3 to 6 make one layer. GPT-2 repeats it 12 times, Qwen3 28 times and Granite 32 times.

Step 7: from vector to probabilities#

After the last layer comes one more norm. Then the vector is compared with every row of the embedding table from Step 2. All three models reuse that same table here, so the score for " Alice" is how well the final " to" vector matches the " Alice" row. Granite divides the scores by 3 first. Softmax turns the scores into probabilities.

# Qwen3 and Granite; GPT-2 uses LayerNorm and its wte table
x = rms_norm(x, sd["model.norm.weight"], eps)
logits = x @ sd["model.embed_tokens.weight"].T     # one score per vocabulary row (Granite: then / 3)

The top two are " Alice" and " her" in GPT-2 (0.776, 0.057), " Alice" and " the" in Qwen3 (0.980, 0.006), and " Alice" and " him" in Granite (0.986, 0.006).

Where the name appears#

Step 7 can be applied after any layer, not only the last. We take the " to" vector as it leaves layer i, run it through the final norm and the embedding table, and read the probability of " Alice":

for i, x in enumerate(layer_outputs):      # x: the " to" vector after layer i
    p = (rms_norm(x, sd["model.norm.weight"], eps) @ sd["model.embed_tokens.weight"].T).softmax(-1)
    print(i, p[alice_id].item())           # GPT-2: LayerNorm and wte; Granite: scores / 3 first

That shows what the model would say if it stopped after layer i. This is called the logit lens. It is rough, because middle layers were not trained to be read this way, but it shows when an answer forms.

In all three, " Alice" appears late, in the last quarter of the layers: at layer 9 of 12 in GPT-2, 24 of 28 in Qwen3 and 27 of 32 in Granite. Just before that, the top word is a pronoun: " him" in GPT-2 at layers 6 to 8 and " her" in Qwen3 at layer 23. The models seem to settle on "a person goes here" first and fill in which person later.

In Granite, layer 27 is an attention layer. In the mixing table above, it is also the layer where " to" takes a visible share from " Alice". It looks as if the attention layer fetches the name. The dashed line says otherwise: with all four attention layers switched off, " Alice" appears one layer later, after Mamba2 layer 28, and ends at 0.988.

Switching layers off#

To switch a mixing step off, we let it compute as usual and multiply its output by zero. In Step 5 the layer then adds nothing (in GPT-2, only the fixed bias of the attention output projection is still added), so the vector passes through almost unchanged, and the layer's FFN still runs. In our code every attention layer calls the same attention function, so we swap that function for one that returns zeros:

orig = hf.attention
hf.attention = lambda q, k, v, scale: orig(q, k, v, scale) * 0   # every attention layer adds 0
logits = hf.granite_forward(sd, cfg, ids, {})
hf.attention = orig                                              # put it back

To switch off one layer instead of all, the replacement counts its calls and zeroes only the j-th one. Mamba2 layers are switched off the same way through the mamba2 function. After every run we checked that the number of zeroed calls matched the plan (for example 4 for all attention layers, 28 for all Mamba2 layers). The question is always the same 18-token sentence; we read P(" Alice") at the last token.

One layer at a time, the answer mostly survives. In GPT-2, switching off any one of the 12 attention layers leaves " Alice" on top, and the lowest probability is 0.427 (layer 5). In Qwen3, 27 of 28 single switches leave it at 0.843 or higher. The exception is layer 0: without it Qwen3 answers " Web", and Granite without its first Mamba2 layer answers " their". Those two layers seem to be needed for the model to work at all, not specifically for the name.

In Granite, switching off all 28 Mamba2 layers gives 0.000, and the top word becomes " *_": the model stops producing sense at all. Switching off only the 14 Mamba2 layers from layer 16 on gives 0.0001 and the answer " the". Switching off all four attention layers gives 0.988. On this sentence the name does not need attention. The test cannot tell whether the Mamba2 layers carry the name or simply keep the model working.

Making the distance longer#

In the 18-token sentence, "Alice" is at position 4 and the question ends at position 17, so the name is only 13 tokens back. To test a longer gap without giving the model anything new to copy, we built each input from three parts:

• Head: "In the morning, Alice gave Bob a book."
• Filler: one paragraph with no people and no book in it, so it holds no hint about the answer.
• Tail: "Later, Bob gave the book back to"

This is the filler, word for word:

The sky was grey and a light rain fell over the town. The bakery on the corner opened late, and the smell of bread drifted down the street. A bus stopped, waited, and pulled away. In the park, the benches were wet and nobody sat on them. The river was high after a week of rain, and the old bridge creaked when the wind picked up. Shops put out their signs one by one.

We put the same filler paragraph in 0, 1, 4 or 16 times. The question and the right answer never change; only the gap grows. The token counts came out the same in all three tokenizers:

text = HEAD + FILLER * n + TAIL     # n = 0, 1, 4, 16

For each input we read P(" Alice") at the last token. For Granite we did this twice: once as trained, and once with all four attention layers switched off as above.

Already at 105 tokens, one filler paragraph, Granite without attention falls to 0.138 and answers " the". At 1,410 tokens it is 0.002. Granite as trained stays at 0.96 or higher at every length, and so does Qwen3, which has only attention layers.

So with the four attention layers on, the name survives every length we tried, and with them off it is lost from 105 tokens on. We did not measure the state or the attention weights on the long inputs, so the reason is our reading: a fixed-size state has to fit everything into the same space and fades old content a little at every step, while attention can read "Alice" directly at any distance. That matches the division of work a hybrid design aims for: many layers with a cheap fixed-size state, and a few that can still look back directly.

What this does and does not show#

• One sentence and one filler paragraph. A different sentence, or filler with names in it, could move these numbers.
• Switching a layer off is blunt. The later layers never saw a zero there during training, so a drop shows that the layer is needed, not exactly what it does.
• The logit lens and the mixing tables describe; only the switch-off tests test.
• These are three small base models on a CPU in float32. Larger models of the same families may behave differently.

The three models, cell by cell#

The frame is the same in all three; what differs is the cells in this table. The next post goes through those cells across more designs, from the original Transformer to Mamba and other hybrids. If you want it in your inbox, the newsletter form is below.