Where Decision Models Read Their Answer: laya, DiffusionGemma, Kev, CLM and a Plain Classifier in Code
Five ways to turn a customer message and 77 intent labels into a probability per label, followed through the code of each. On the same 154 BANKING77 messages they scored from 3.2% to 90.3%. laya cuts every label to three tokens, and giving it more room lifts it from 37.0% to 46.1%; the others keep labels whole, place the options differently, and read the score elsewhere.

A bank customer types "What is the top-ups limit?" and a decision model has to pick one of 77 intents. The right one is top_up_limits. laya, a 421M decision model, picked top_up_by_cash_or_cheque. When we fed laya's input builder the same 77 labels and counted tokens, both labels came out as the same three tokens, top_up, as did four other top-up intents. Inside laya's input, six different answers were spelled identically.
That one message is a small version of a bigger spread. On the same 154 BANKING77 messages, with label names only, the five systems this post opens scored from 3.2% to 90.3%. Jev is listed for reference:
| System | Accuracy on the same 154 messages |
|---|---|
| laya, default settings | 37.0% |
| DiffusionGemma through openjev, one read | 66.9% |
| Kev-9B (BANKING77 is in its training data) | 82.5% |
| CLM, label names only | 3.2% |
| MiniLM embeddings + logistic regression, trained on 10,003 labelled messages | 90.3% |
| *Reference: Jev 1.13 (TypeSafe API, internals not public)* | *76.0%* |
These numbers come from our earlier posts (laya, openjev and the classifier, Kev and Jev, CLM). The Kev row needs its caveat in the same breath: Kev was trained on BANKING77, so this task is not new to it the way it is to laya or openjev.
This post opens the code of each system and follows that one message until it becomes a probability. The architectures are public for all of them except Jev, so the differences can be read straight from the source.
What a decision model does
A decision model does not write an answer. You send it a state (here, the customer's message), a question, and a fixed list of options. It sends back one probability per option. Our ten-minute explainer covers why that is useful: you act on the confident answers and pass the rest to a slower model or a person.
Every system below has to solve the same two problems. It has to show the model the options somehow, and it has to read one score per option from somewhere inside the network. The five systems answer those two questions differently, and with 77 options the first one becomes especially important.

laya: every option inside one input, one score per [MASK]
laya is a ModernBERT-large encoder with a small head on top. It builds one sequence per question in laya/common.py:
[CLS] choice question: <instructions> [SEP] [MASK] option0 [MASK] option1 ... [SEP] <state> [SEP]Each option starts with its own [MASK] token. After the encoder runs, the head picks up the hidden vector at every [MASK] position, turns each into a single number, and takes a softmax over the options:
idx = marker_pos.clamp(min=0)[:, :, None].expand(-1, -1, h.size(-1))
m = torch.gather(h, 1, idx)
logits = self.scorer(m).squeeze(-1).float()
Because this is an encoder, every [MASK] can attend to the message and to every other option, which is the strength of the design. The cost is that all the options have to fit into the instruction-and-options part of the input, and that part has a fixed size, head_max_len, which is 192 tokens in the English checkpoint. When the options don't fit, this block cuts every option to the same length:
opt_budget = head_max_len - sum(len(o) for o in opt_ids)
if opt_budget < 16:
per = max(4, (head_max_len - 16) // max(1, len(opt_ids)))
opt_ids = [o[:per] for o in opt_ids]
opt_budget = head_max_len - sum(len(o) for o in opt_ids)
head_ids = head_ids[: max(8, opt_budget)]With 77 options, per hits its floor of 4: one [MASK] and three label tokens. We ran this function with laya's own tokenizer on the 77 BANKING77 labels and the message above (tokenizer only, no model):
- The labels are 3 to 17 tokens long, 5 at the median, 476 in total. The budget is 192.
- 64 of the 77 labels get cut. After cutting, only 67 distinct labels remain. Six labels become
top_up, three becomecard_payment. - The instruction keeps 8 tokens and now reads "choice question: Which intent does this banking".
- The options take 308 positions, 77 times four, well past the 192 budget, because the floor of 4 overrides it. The message itself is 8 tokens and still fits.

Raising the budget to 512 (with max_len 1024) leaves the instruction whole and cuts 37 labels instead of 64, with 72 distinct labels left. In our run that setting answered top_up_limits correctly for the message above, and accuracy rose from 37.0% to 46.1%. Shortlisting to 20 labels with an embedding model first, the other fix laya's README suggests, reached 59.1%, though MiniLM alone already picked the right label for 87 of the 154 messages (laya on the shortlist got 91). laya's own benchmark notes say it plainly: "Keep choice questions under ~20 options" (BENCHMARKS.md).
DiffusionGemma: leave one blank on the answer sheet and read it
openjev wraps a different idea, taken from the vLLM pull request for DiffusionGemma-26B-A4B that we tested in our DiffusionGemma post. A diffusion language model fills a whole block of text at once instead of one token at a time. openjev writes the answer sheet onto that block in advance and leaves exactly one position blank (openjev/engine.py):
canvas = list(template) + [TURN_CLOSE]
canvas += [PAD] * (self.canvas_width(template) - len(canvas))
for s in slots:
canvas[s["pos"]] = rng.randrange(VOCAB)The template is q1: X. The options themselves go into the system prompt as a legend, A: card_arrival, B: card_linking and so on. One denoising step later, openjev reads the logits at the blank, keeps only the token ids of the legend symbols, and takes a softmax over those.

The catch is that each option has to be one token at that one position, so the label is the symbol, never the intent name. openjev checks every label and refuses the request otherwise:
if len(e) != len(base) or len(diffs) != 1 or (pos is not None and diffs[0] != pos):
raise SchemaError(f"question {q['key']!r}: labels do not share one template slot")For 77 options this avoids laya's problem. The intent names are never cut, because they sit in the prompt, which has room for all of them, and the answer is still one position. The model's job becomes reading a 77-line legend and writing one letter, and openjev got 66.9%. The legend is still a prompt, though, and wording matters. In that post, vLLM's own example server, which uses the same one-slot read but stops at 26 options, fell from 54.5% to 31.8% when we added one sentence describing the input.

Kev: options as spans in a decoder, scored by pointing
Kev is a Qwen3.5 base model (0.8B, 4B or 9B) with a LoRA adapter and a small head; our first Kev post measured it on the tasks it trained on. It packs the request into one sequence, reusing five rarely used Qwen special tokens as markers (kev/model.py):
<state> message <q> instructions <opt> card_arrival </opt> <opt> card_linking </opt> ... <decide>Kev does not read the probability of a label token. Instead, it takes the hidden vector at the end of each option, the </opt> token, and compares it with the hidden vector at <decide>, which comes after every option and so has seen all of them:
def forward(self, h_decide, h_opts): # [d], [K,d] -> logits [K]
z = (self.k(h_opts) @ self.q(h_decide)) * self.scale
return z if self.training or self.temperature == 1.0 else z / self.temperature
That is one score per option, like laya, but the options are not squeezed into a fixed share of the input. Every label goes in whole, and each question's row, message included, can run to 8,192 tokens when served. For BANKING77 that means all 77 names arrive intact. The temperature that softens the scores is stored in the checkpoint; for the 9B revision we tested (2629c06) it is 2.297.
Kev-9B scored 82.5% on our 154 messages, ten more right than Jev, though that gap is not statistically clear. Part of it is training data: BANKING77 is one of the tasks Kev was trained on. On three task sets Kev had not seen, Jev came out ahead in two (Kev vs Jev), so the layout explains why 77 options are not a problem for Kev, not why it beats the others.

CLM: two embeddings and a cosine
CLM takes the opposite route to laya. The message and each option never share an input. A Qwen3-8B encoder embeds the message with the question's instructions, embeds every option separately, a small trained head projects both, and the score is the scaled cosine between them (src/clm/engine.py):
cos = za[k:k + len(texts)] @ zq[i]
k += len(texts)
answers[qid] = answer_from_logits(questions[qid], keys, (scale * cos / temperature).tolist())
Option embeddings can be cached, so adding options is cheap, and nothing is ever cut. But the option is now judged by its text alone, and top_up_limits as a bare label is a poor sentence to compare a question against. With label names only, CLM scored 3.2% on our 154 messages, where random choice gives 1.3% (decision models and confidence). One-line descriptions helped on the tasks where we had them, which fits how it was trained, on answer sentences rather than one-word labels.
The plain classifier: the options are never in the input
The classifier we used as a baseline does not show the model the options at all. These lines are from our benchmark script, with the timing calls taken out:
m = SentenceTransformer(a.model, device=a.device) # all-MiniLM-L6-v2
Xtr = m.encode([r["text"] for r in train], batch_size=256, normalize_embeddings=True, show_progress_bar=False)
clf = LogisticRegression(max_iter=2000, C=10.0)
clf.fit(Xtr, [r["category"] for r in train])
Each of the 77 intents is a row of weights learned from labelled messages, 10,003 of them. Nothing about the options competes for input space, because the input is only the message. That is why 77 options cost it nothing, and it scored 90.3% on CPU. The price is the training set. Without labelled examples this option does not exist, which is the case decision models were built for.
The five systems side by side

| Where the score is read | Options in the input | Long labels cut at 77 options | Needs labelled examples | 154 messages | |
|---|---|---|---|---|---|
| laya | [MASK] before each option | yes, in a shared 192-token budget | yes, to 3 tokens | no | 37.0% |
| DiffusionGemma (openjev) | one blank answer position | names in the prompt, answer is one symbol | no | no | 66.9% |
| Kev-9B | </opt> of each option, against <decide> | yes, each label whole | no | no (but trained on BANKING77) | 82.5% |
| CLM | cosine between two embeddings | each option embedded on its own | no | no | 3.2% |
| MiniLM + logistic regression | learned weight row per intent | no | n/a | yes, 10,003 | 90.3% |
All five used the same 154 messages and label names without descriptions. They differ in model size, training data and hardware, so the table shows how each system handles the options, not a ranking of architectures.
Jev is the one system we cannot open. TypeSafe has said it uses a new model architecture, a parallel sampler and a training method called RLCD, and that it produces all outputs in one request. Its internals are not public, so we have left it off the comparison table.
Before you send 77 options
If you plan to use a decision model on a long option list, find out where your options go. If they share a fixed budget inside the input, as with laya, count the tokens yourself before trusting the result. Here is the check we used, with laya installed:
from transformers import AutoTokenizer
from laya.common import build_sequence
tok = AutoTokenizer.from_pretrained("convaiinnovations/laya", subfolder="tokenizer")
question = {"t": "choice", "ins": instructions, "crit": {l: None for l in labels}}
# max_len and head_max_len: use your checkpoint's config values
seq, markers = build_sequence(tok, message, question, max_len=512, head_max_len=192)
ends = markers[1:] + [seq.index(tok.sep_token_id, markers[-1])]
kept = [tok.decode(seq[m + 1:e]).strip() for m, e in zip(markers, ends)]
print(len(set(kept)), "distinct labels out of", len(kept))If you already have a loaded agent, agent.tok, agent.cfg["max_len"] and agent.cfg["head_max_len"] give the same three values.
If that number is smaller than your label count, some of your answers are identical from the model's point of view. Shortlist first, shorten the labels so they differ in their first tokens, or pick a system that keeps labels whole.
Code read at these commits: laya 010bace, openjev 3e296f0, Kev 09ff745, CLM cca045f. Accuracy figures are from the result files of the linked posts; token counts use laya's own tokenizer on the 77 BANKING77 labels.
