From Autocomplete to Assistant: Next-Token Prediction

By the end of this lesson you will be able to write the objective every model in this course was trained on as a formula and compute it for one sentence on your own machine; say what a forward pass produces, one probability for each of the 151,936 entries in the Qwen3 vocabulary, and read one; predict in numbers what temperature, top-k and top-p do to that distribution before a token is drawn; show a chat turn as the token sequence the model actually receives; explain, with a measurement, why fluent and false output is the objective working rather than a defect added later; and turn the phrase “the weights encode a fact” into a number you can compute. Everything in Parts 3 to 27 sits on top of these facts.

One objective

A language model is trained on one task, repeated across a very large amount of text: given the tokens so far, produce a probability for every token that could come next. Written down, with t_1 … t_N the tokens of one document and p(t_i | t_1 … t_{i-1}) the probability the model gave to the token that actually stood at position i:

loss(document) = -(1 / (N − 1)) × sum over i = 2 … N of ln p(t_i | t_1 … t_{i-1})

The first token has nothing before it and is not scored, which is why seven tokens give six losses in the table below. That is the cross-entropy of Part 1’s loss lesson, averaged over every scored position of every document, and nothing else. The label at each position is the next token of the text itself, so a document of N tokens is N − 1 labelled examples that nobody had to label, which is why the training set can be the size it is. Gradient descent nudges every parameter to raise p for the true continuation a little, across trillions of positions, and the structure the parameters end up encoding is whatever lowered that number.

One forward pass produces every one of those predictions at once. For the seven tokens of The capital of France is Paris., with ids from the Qwen3 tokeniser, a file that Qwen3-0.6B, Qwen3-1.7B and Qwen3-8B (all Apache-2.0) share byte for byte according to the Hub’s file listings on 2026-09-12:

Position i	Tokens given	True next token, the label	Label id
0	The	Ġcapital	6722
1	The capital	Ġof	315
2	The capital of	ĠFrance	9625
3	The capital of France	Ġis	374
4	The capital of France is	ĠParis	12095
5	The capital of France is Paris	.	13
6	The capital of France is Paris.	nothing follows; this row is ignored	—

The logits tensor of that pass has the shape (1, 7, 151936), one row per position, and row i is scored against the token at i + 1. In Transformers you pass the same ids as labels; the loss code at the pinned transformers 5.16.1 · verified 2026-09-08 tag shifts them for you (the comment in loss_utils.py reads “Shift so that tokens < n predict n”) and ignores any label set to -100, the hook that lets post-training score only the assistant’s tokens; Part 13 switches it on through TRL’s completion_only_loss and assistant_only_loss settings, and TRL’s own docstring at the pinned TRL 1.12.0 · verified 2026-09-08 tag says that with the first set to true “the labels for the prompt part are set to -100”.

This snippet and the two later ones that load a model need the environment step and the download from tasks 1 and 2 of this part’s lab; run them after the lab, or now if it is done. Save each block to a file in ~/llm-course and run it with that environment active (Track S: /workspace/course, with the container substitutions the lab’s Requirements tab gives). The outputs shown are the author’s dry run on Qwen3-0.6B, the lab’s reduced path, on a CPU in BF16 with transformers 5.16.1 on 2026-09-12; on Qwen3-1.7B every probability will differ and every id and shape will not. The Loading weights line is Transformers’ own progress bar.

RunnableAll tracks

per-token-loss.py

"""Score one sentence the way pretraining scores it: -ln p of each token, given the tokens before it."""
import math
import sys
from pathlib import Path

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_dir = Path(sys.argv[1] if len(sys.argv) > 1 else "~/llm-course/models/qwen3-1.7b").expanduser()
tokeniser = AutoTokenizer.from_pretrained(model_dir)
model = AutoModelForCausalLM.from_pretrained(model_dir, dtype=torch.bfloat16)
model.eval()

text = "The capital of France is Paris."
ids = tokeniser(text, return_tensors="pt")["input_ids"]
with torch.no_grad():
    out = model(input_ids=ids, labels=ids)          # labels are the same ids; the shift is done inside
log_probs = torch.log_softmax(out.logits[0].float(), dim=-1)   # one row per position, 151,936 wide

pieces = tokeniser.convert_ids_to_tokens(ids[0].tolist())
print(f"{'pos':>3s}  {'given (text so far)':<34s}  {'true next':<10s}  {'p(true next)':>12s}  {'-ln p':>6s}")
losses = []
for i in range(len(pieces) - 1):
    loss = -log_probs[i, ids[0, i + 1]].item()
    losses.append(loss)
    print(f"{i:3d}  {tokeniser.decode(ids[0, : i + 1])!r:<34s}  {pieces[i + 1]!r:<10s}  {math.exp(-loss):12.6f}  {loss:6.3f}")
mean = sum(losses) / len(losses)
print(f"\nmean of the {len(losses)} losses: {mean:.4f}   model's own loss: {out.loss.item():.4f}   exp(mean) = {math.exp(mean):.2f} equally likely options per token")

Output — what you should see

Loading weights: 100%|██████████| 311/311 [00:00<00:00, xxxx.xxit/s]
pos  given (text so far)                 true next   p(true next)   -ln p
  0  'The'                               'Ġcapital'      0.000003  12.785
  1  'The capital'                       'Ġof'           0.635943   0.453
  2  'The capital of'                    'ĠFrance'       0.020778   3.874
  3  'The capital of France'             'Ġis'           0.770903   0.260
  4  'The capital of France is'          'ĠParis'        0.638311   0.449
  5  'The capital of France is Paris'    '.'             0.525538   0.643

mean of the 6 losses: 3.0773   model's own loss: 3.0773   exp(mean) = 21.70 equally likely options per token

Read the table against the formula. At position 0 the model has one token of context and gave the token that actually followed a probability of three in a million, a loss of 12.8, above the 11.93 that spreading probability evenly over 151,936 entries would cost: one token is nearly no information, and the weights preferred other continuations of The. From position 3 onwards the losses are small. The mean, 3.08 nats, matches the loss the model returned to four decimals, which confirms that the model’s number is exactly this arithmetic, and exp(3.08), about 22, is the number of equally likely options the model was choosing among on average: its perplexity on this sentence. The curves in Part 3’s pretraining lesson are this quantity averaged over billions of positions.

Nothing else is in the objective. Not truthfulness, not helpfulness, not refusing dangerous requests. Those come from later training stages, which Part 3’s post-training lesson covers, and they are adjustments to a model whose foundation is this one prediction task.

The output is a distribution, not a word

Run a model over a piece of text and, for the position after the last token, it produces one number for every entry in its vocabulary: for Qwen3-8B, whose config.json gives vocab_size as 151936, that is 151,936 numbers. They come from one matrix multiplication: the last layer’s output vector, hidden_size wide, times an unembedding matrix of shape vocab_size × hidden_size, the counterpart of the embedding matrix the embeddings lesson opens. Arithmetic from the two config.json files, not a measurement:

Model	hidden_size	vocab_size	Unembedding entries	Bytes in BF16
Qwen3-1.7B	2,048	151,936	311,164,928	622,329,856
Qwen3-8B	4,096	151,936	622,329,856	1,244,659,712

The lab’s second safetensors shard of Qwen3-1.7B is 622,329,984 bytes: that tensor plus a 128-byte header, and task 4 of the lab explains why it is shipped although the config says tie_word_embeddings is true. For a small model the unembedding is a large share of the whole, about 18 per cent of the 1,720,574,976 parameters the lab’s task 6 reports, and every generated token pays for it once.

The raw numbers are logits, z. They are not probabilities: they can be negative and they do not sum to one. A softmax turns them into a probability distribution:

p_i = exp(z_i) / sum over all j of exp(z_j)

Part 1 worked the four-logit case by hand. The result is the model’s answer: not “the next word is Paris” but a probability for every token. In the hallucination section’s run of the same prompt below (top-5 share 0.723), after The capital of France is, ĠParis took 0.638, the next four tokens 0.085 between them, and the remaining 151,931 entries shared 0.277: even with one obvious answer, the output is a distribution. How spread out it is has a number, the entropy H = -(sum over i of p_i × ln p_i), in nats, and exp(H) is the number of equally likely options that would carry the same uncertainty. The training loss is the same formula evaluated at the true token instead of averaged over the model’s own; the hallucination section measures H for three prompts.

Generating one token, then the next

1. 1TextThe prompt, plus everything generated so far.
2. 2TokeniseSplit into token ids from the model’s vocabulary.
3. 3Forward passOne logit per vocabulary entry, for the position after the last token.
4. 4SoftmaxLogits become probabilities that sum to one.
5. 5ChooseGreedy takes the highest; sampling draws from the distribution.
6. 6AppendAdd the chosen token and go back to step 2, until a stop token is chosen or the budget runs out.

The loop that produces all text. Steps 2 to 4 are deterministic and the same for every model in this course; step 5 is the only place randomness can enter. Task 6 of the lab stops after step 4 and prints the distribution.

The loop is why generation is sequential: token seventeen cannot be computed until token sixteen has been chosen and appended. Part 3’s inference lesson names this loop decode and explains why its speed is set by memory bandwidth rather than arithmetic.

Sampling turns probabilities into text

Step 5 is a choice. The Transformers documentation at the pinned tag describes greedy search as “the default decoding strategy”, which “selects the next most likely token at each step”, and says it “breaks down when generating longer sequences because it begins to repeat itself”. Sampling, enabled with do_sample=True, “randomly selects a token based on the probability distribution over the entire model’s vocabulary”, so that “every token with a non-zero probability has a chance to be selected”.

A draw is a cumulative sum. Add the probabilities up in vocabulary order to get a running total that ends at 1, draw one uniform random number u in [0, 1), and take the first token whose running total exceeds u. A token with probability 0.638 owns 63.8 per cent of that line and is drawn that often; a seed fixes the sequence of u values, which is why a seeded run repeats and an unseeded one does not. Three settings reshape the distribution before the draw, and every engine in this course exposes them:

• Temperature T divides the logits: p_i = exp(z_i / T) / sum over j of exp(z_j / T), which is the same as raising every probability to the power 1/T and renormalising. Below one the gaps grow and the leader takes more; above one they shrink and the tail gets a real chance; as T approaches zero the draw becomes greedy.
• Top-k keeps the k highest-probability tokens, zeroes the rest and renormalises.
• Top-p, nucleus sampling, sorts by probability, keeps the smallest set whose probabilities reach p, and renormalises; how many survive depends on how peaked the distribution is.

The snippet applies all three to six stated logits, shaped like the top of the dry run’s list, and makes one seeded draw. It needs only numpy, which Part 1’s environment installed alongside PyTorch, so it runs now.

RunnableAll tracks

sampling.py

"""Softmax, temperature, top-k, top-p and one seeded draw, on six stated logits."""
import numpy as np

tokens = [" Paris", " located", " the", " a", " home", " Lyon"]
logits = np.array([17.5, 14.3, 14.1, 13.8, 12.6, 11.9])


def softmax(z):
    e = np.exp(z - z.max())          # subtracting the max changes nothing and avoids overflow
    return e / e.sum()


def truncate(p, top_k=None, top_p=None):
    keep = np.ones_like(p, dtype=bool)
    order = np.argsort(-p)           # ranks, most probable first
    if top_k is not None:
        keep[order[top_k:]] = False
    if top_p is not None:
        running = np.cumsum(p[order])
        first_past = int(np.argmax(running >= top_p))   # smallest set whose mass reaches top_p
        keep[order[first_past + 1:]] = False
    q = np.where(keep, p, 0.0)
    return q / q.sum()               # renormalise what survived


print(f"{'token':>10s} {'logit':>6s} {'T=1.0':>7s} {'T=0.5':>7s} {'T=1.5':>7s} {'top-k=3':>8s} {'top-p=0.9':>9s}")
cols = [softmax(logits / t) for t in (1.0, 0.5, 1.5)]
cols.append(truncate(cols[0], top_k=3))
cols.append(truncate(cols[0], top_p=0.9))
for i, tok in enumerate(tokens):
    print(f"{tok!r:>10s} {logits[i]:6.1f} " + " ".join(f"{c[i]:7.4f}" if j < 3 else f"{c[i]:8.4f}" if j == 3 else f"{c[i]:9.4f}" for j, c in enumerate(cols)))

p = cols[0]
rng = np.random.default_rng(seed=7)
u = rng.random()                     # one uniform number in [0, 1)
cumulative = np.cumsum(p)
chosen = int(np.argmax(cumulative > u))
print(f"\ndraw: u = {u:.4f}; cumulative = {np.round(cumulative, 4).tolist()}")
print(f"the first token whose cumulative probability exceeds u is {tokens[chosen]!r}")
print(f"greedy would take {tokens[int(np.argmax(p))]!r} every time")

Output — what you should see

     token  logit   T=1.0   T=0.5   T=1.5  top-k=3 top-p=0.9
  ' Paris'   17.5  0.9009  0.9966  0.7305   0.9310    1.0000
' located'   14.3  0.0367  0.0017  0.0865   0.0379    0.0000
    ' the'   14.1  0.0301  0.0011  0.0757   0.0311    0.0000
      ' a'   13.8  0.0223  0.0006  0.0620   0.0000    0.0000
   ' home'   12.6  0.0067  0.0001  0.0279   0.0000    0.0000
   ' Lyon'   11.9  0.0033  0.0000  0.0175   0.0000    0.0000

draw: u = 0.6251; cumulative = [0.9009, 0.9376, 0.9677, 0.99, 0.9967, 1.0]
the first token whose cumulative probability exceeds u is ' Paris'
greedy would take ' Paris' every time

At T = 0.5 the leader goes from 0.90 to 0.997 and the sixth token to nothing; at 1.5 the leader drops to 0.73 and the tail gets a real chance. Top-k of 3 zeroes three entries however the mass was spread. Top-p of 0.9 kept only Paris, because Paris alone reaches 0.9; on the flat distribution measured in the hallucination section the same setting keeps hundreds, which is why publishers recommend top-p and why a fixed top-k is blunt. The seeded draw gave u = 0.6251, inside Paris’s 0.90 of the line; any u above 0.9009 would have chosen a later token. The order in which an engine applies the three matters, because truncating then flattening is not flattening then truncating; Part 6’s sampling lesson reads llama.cpp’s printed chain and adds min-p, penalties and seeds.

Sampling is why two runs differ. If the top token holds probability q at every step and each draw is independent, the chance that n sampled tokens all equal greedy’s is q^n, arithmetic rather than a measurement:

q at every step	n = 10	n = 100	n = 500
0.9	0.349	0.00003	effectively 0
0.99	0.904	0.366	0.0066
0.999	0.990	0.905	0.606

A 500-token answer sampled at the publisher’s settings will differ from a greedy one somewhere with near certainty, and from its own rerun too. Those settings ship in generation_config.json, a file the three Qwen3 sizes share byte for byte:

Key	Value	What it does to the draw
do_sample	true	draw rather than take the maximum
temperature	0.6	sharpen: every probability raised to the power 1/0.6, about 1.67, and renormalised
top_k	20	at most twenty candidates survive
top_p	0.95	fewer than twenty if the leaders already reach 0.95
eos_token_id	[151645, 151643]	stop when the end-of-turn or the end-of-text token is drawn

The Qwen3-1.7B model card is blunt: for thinking mode it says “DO NOT use greedy decoding, as it can lead to performance degradation and endless repetitions”, and for non-thinking mode it suggests temperature 0.7 and top-p 0.8 with the same top-k. Which to use is a decision about what you are doing:

You want	Use	Why
A rerun that reproduces a measurement (the lab’s task 7)	greedy, do_sample=False, or sampling with a fixed seed	removes the draw, or fixes it
Answers from a chat model	the card’s settings	the distribution it was post-trained to be sampled from
Several different drafts	a temperature above the card’s, same truncation	flatter distribution, more variety at every step
Output that must parse, JSON or a schema	constrained decoding, not the sampler	Part 10’s structured-output lesson

Beam search is the third documented family: it “keeps track of several generated sequences (beams) at each time step” and picks the sequence with the highest overall probability. The documentation calls it “best suited for input-grounded tasks, like describing an image or speech recognition”, and it is rarely the right choice for chat.

✕Common mistake

A common misreading is that temperature controls how “creative” or how “accurate” a model is. It controls neither. It reshapes a distribution the model already produced: in the table above the wrong token Lyon has the same rank at every temperature, and if a wrong token had been the leader, T = 0.5 would have made it more certain, not less. Part 16 measures what settings do to a task instead of guessing.

Chat is still next-token prediction

An assistant that answers questions looks like a different kind of program from an autocomplete. It is not. The chat-template documentation at the pinned tag puts it plainly: “All causal LMs, whether chat-trained or not, continue a sequence of tokens”, and the list of role and content dictionaries you pass to a chat model “get converted to a token sequence, often with control tokens like <|user|> or <|assistant|> or <|end_of_message|>, which allow the model to see the chat structure”.

The converter is the chat template, a Jinja program stored under the chat_template key of tokenizer_config.json, 4,168 characters long for Qwen3. It renders the list to one string, the tokeniser turns the string into ids, and the same loop runs. The snippet renders a two-turn conversation and marks which tokens are the template’s control tokens and which are text:

RunnableAll tracks

chat-tokens.py

"""What a two-turn conversation becomes on its way to the model: one token sequence, with the control tokens marked."""
import sys
from pathlib import Path

from transformers import AutoTokenizer

model_dir = Path(sys.argv[1] if len(sys.argv) > 1 else "~/llm-course/models/qwen3-1.7b").expanduser()
tokeniser = AutoTokenizer.from_pretrained(model_dir)

messages = [
    {"role": "system", "content": "Answer in one word."},
    {"role": "user", "content": "What is the capital of Brazil?"},
    {"role": "assistant", "content": "Brasília."},
    {"role": "user", "content": "And of Australia?"},
]
ids = tokeniser.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True)["input_ids"]
special = set(tokeniser.all_special_ids)
content = sum(len(tokeniser(m["content"])["input_ids"]) for m in messages)

print(f"eos token: {tokeniser.eos_token!r} = id {tokeniser.eos_token_id}")
print(f"{len(ids)} tokens in total; {content} of them are the four messages' text, {len(ids) - content} are added by the template\n")
for pos, token_id in enumerate(ids):
    kind = "control" if token_id in special else "text"
    print(f"{pos:3d}  {token_id:7d}  {kind:<8s} {tokeniser.convert_ids_to_tokens(token_id)!r}")

Output — what you should see

eos token: '<|im_end|>' = id 151645
43 tokens in total; 20 of them are the four messages' text, 23 are added by the template

  0   151644  control  '<|im_start|>'
  1     8948  text     'system'
  2      198  text     'Ċ'
  3    16141  text     'Answer'
  4      304  text     'Ġin'
  5      825  text     'Ġone'
  6     3409  text     'Ġword'
  7       13  text     '.'
  8   151645  control  '<|im_end|>'
  9      198  text     'Ċ'
 10   151644  control  '<|im_start|>'
 11      872  text     'user'
 12      198  text     'Ċ'
 ...
 20   151645  control  '<|im_end|>'
 21      198  text     'Ċ'
 22   151644  control  '<|im_start|>'
 23    77091  text     'assistant'
 24      198  text     'Ċ'
 25     6828  text     'Br'
 26      300  text     'as'
 27    75372  text     'ÃŃlia'
 28       13  text     '.'
 29   151645  control  '<|im_end|>'
 30      198  text     'Ċ'
 31   151644  control  '<|im_start|>'
 32      872  text     'user'
 ...
 38   151645  control  '<|im_end|>'
 39      198  text     'Ċ'
 40   151644  control  '<|im_start|>'
 41    77091  text     'assistant'
 42      198  text     'Ċ'

control marks the special-token ids; the role words and newlines the template inserts are ordinary vocabulary tokens, which is why 23 tokens are template-added but only 9 rows are marked control: two markers for each of the four messages and the <|im_start|> that opens the reply. Every message costs five template tokens, the start marker, the role word, a newline, the end marker and a newline, and the last three tokens are the generation prompt that add_generation_prompt=True appends: an open assistant turn for the model to continue. The documentation says what happens without it: “the model may get confused and do something strange, like continuing the user’s message instead of replying to it”. Two consequences follow from the listing and are easy to miss in a chat window.

The assistant’s earlier reply is now prompt. Positions 25 to 28 are Brasília., marked text like the user’s words, because that is what they are: the model has no memory of having produced them, and every turn re-reads the whole history through prefill. With a ten-token system message and thirty-token messages, the prompt at turn k holds one system message, k user messages and k − 1 replies, so 15 + (2k − 1) × 35 + 3 tokens, arithmetic from the template’s counts:

Turn	Messages in the prompt	Prompt tokens
1	2	53
5	10	333
10	20	683
20	40	1,383

Part 3 prices what that costs in memory and Part 10 shows engines reusing the unchanged prefix.

Stopping is a token. The turn ends when the draw picks <|im_end|>, id 151645, the eos_token of the tokeniser and the first entry of eos_token_id in the generation config. There is no separate “I am finished” signal: the end-of-turn token has a probability like any other, which is why a run with too small a max_new_tokens is cut off mid-sentence and why task 7 of the lab reports whether generation “stopped on end-of-turn”.

Different models use different control tokens, and the documentation is explicit that “with the wrong control tokens, these models would have drastically worse performance”; a model served with another family’s template is being asked a question in a format it never saw during training, which is why Parts 6 and 7 spend time on templates. What post-training changed is the weights, so that after <|im_start|>assistant and a newline an answer is the likely continuation. The two model cards state the difference task 7 of the lab runs: Qwen3-1.7B lists its training stage as pretraining and post-training and ships with this template, while Qwen3-1.7B-Base (Apache-2.0) lists pretraining only. Same architecture, same tokeniser, same objective during pretraining; hand the base checkpoint a question as plain text and its likeliest continuation is another question. Part 3’s post-training lesson explains the stages that make the difference.

Where hallucination comes from

Consider what the loss can see. It is computed from one thing, the probability the model gave to the token that came next in the training text, and from nothing else:

In the loss	Not in the loss
The token that followed in the corpus	Whether the sentence was true
How much probability the model gave it	Whether the entity named exists
The tokens before it, as context	Whether the model has any basis for the continuation
	Whether the reader will act on the output

A citation-shaped string after As shown in the 2019 study by is what usually followed in the training data, so a citation-shaped string is the high-probability continuation. The model has no step in which it checks whether the study exists, because no such step is in the objective and no such signal was in the loss. The snippet measures three prompts: the top five tokens, their share, and the entropy of the whole distribution.

RunnableAll tracks

entropy.py

"""How spread out the next-token distribution is, for three prompts: top 5 tokens and the entropy."""
import math
import sys
from pathlib import Path

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_dir = Path(sys.argv[1] if len(sys.argv) > 1 else "~/llm-course/models/qwen3-1.7b").expanduser()
tokeniser = AutoTokenizer.from_pretrained(model_dir)
model = AutoModelForCausalLM.from_pretrained(model_dir, dtype=torch.bfloat16)
model.eval()

prompts = [
    "The capital of France is",
    "The first author of the paper that introduced this method was",
    "As shown in the 2019 study by",
]
for prompt in prompts:
    ids = tokeniser(prompt, return_tensors="pt")["input_ids"]
    with torch.no_grad():
        logits = model(input_ids=ids).logits[0, -1].float()
    p = torch.softmax(logits, dim=-1)
    entropy = -(p * torch.log(p.clamp_min(1e-12))).sum().item()
    top = torch.topk(p, 5)
    print(f"\n{prompt!r}")
    print("  top 5: " + ", ".join(f"{tokeniser.convert_ids_to_tokens(i)!r} {v:.3f}" for v, i in zip(top.values.tolist(), top.indices.tolist())))
    print(f"  top-5 share {top.values.sum().item():.3f}   entropy {entropy:.2f} nats   exp(entropy) = {math.exp(entropy):.0f} equally likely options")

Output — what you should see

Loading weights: 100%|██████████| 311/311 [00:00<00:00, xxxx.xxit/s]

'The capital of France is'
  top 5: 'ĠParis' 0.638, 'Ġlocated' 0.032, 'Ġthe' 0.023, '...' 0.016, 'Ġ' 0.014
  top-5 share 0.723   entropy 2.42 nats   exp(entropy) = 11 equally likely options

'The first author of the paper that introduced this method was'
  top 5: 'Ġa' 0.104, 'Ġthe' 0.059, 'ĠDr' 0.052, '...' 0.036, 'Ġnot' 0.026
  top-5 share 0.277   entropy 5.68 nats   exp(entropy) = 292 equally likely options

'As shown in the 2019 study by'
  top 5: 'Ġthe' 0.508, 'Ġresearchers' 0.013, 'ĠDr' 0.012, 'Ġa' 0.011, 'ĠSmith' 0.008
  top-5 share 0.551   entropy 4.42 nats   exp(entropy) = 83 equally likely options

The first prompt has one dominant continuation. The second has none: the top five share 0.277 and the distribution is as uncertain as a fair choice among 292 tokens, so whatever name the sampler draws, and the publisher’s settings keep up to twenty candidates in play, it will be drawn from a field of plausible names with no name that the loss ever rewarded for being right. The third sits between: half the mass on Ġthe, then Ġresearchers, ĠDr and ĠSmith, the shape of a citation with the specifics left to the draw. Fluent, confident, false output is therefore the objective working, and no prompt phrasing removes the mechanism. What later training does, as Part 3 explains, is shift the distribution so that hedging and refusal become likely continuations in some contexts. That reduces the rate; it does not change the loop.

The consequence for the rest of the course is architectural. Anything a model outputs that must be true is checked by something outside the model that has the signal the loss never had:

What must be true	The outside check	Where
A statement about documents you hold	Retrieve the passage and put it in the context	Part 10
A current value, a calculation, a lookup	A tool call whose result is fed back as tokens	Part 24
Generated code	Run the tests	Part 25
The rate at which a model gets a class of question wrong	An evaluation over many prompts, scored by a program	Part 16
Everything else	A person	every part

?Reality check

The claim: “the model is lying” or “the model is making things up on purpose.” Neither describes anything in the mechanism: there is no intent in a softmax. The framing this course uses is that the model produced a high-probability continuation and nothing in the system checked it against the world. That framing points at a fix; the other does not.

What “knowing” can honestly mean

There is an honest, narrow sense in which a fact is in a model, and it is a number. Say that the weights encode a fact when, given a prompt that calls for it, the distribution puts high probability on the tokens that state it. For a continuation of several tokens the probability is the product of each token’s probability given everything before it, so its logarithm is a sum, which is the training loss with the sign flipped. The snippet scores candidate continuations that way, and prints the top-ranked next token for each prompt:

RunnableAll tracks

score.py

"""Score candidate continuations: ln p of a whole string is the sum of its tokens' ln p."""
import math
import sys
from pathlib import Path

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_dir = Path(sys.argv[1] if len(sys.argv) > 1 else "~/llm-course/models/qwen3-1.7b").expanduser()
tokeniser = AutoTokenizer.from_pretrained(model_dir)
model = AutoModelForCausalLM.from_pretrained(model_dir, dtype=torch.bfloat16)
model.eval()


def log_probs_for(text: str) -> torch.Tensor:
    ids = tokeniser(text, return_tensors="pt")["input_ids"]
    with torch.no_grad():
        return torch.log_softmax(model(input_ids=ids).logits[0].float(), dim=-1)


def score(prompt: str, continuation: str) -> tuple[float, int]:
    """ln p(continuation | prompt): the sum over the continuation's tokens, each given everything before it."""
    prompt_len = len(tokeniser(prompt)["input_ids"])
    all_ids = tokeniser(prompt + continuation)["input_ids"]
    lp = log_probs_for(prompt + continuation)
    total = sum(lp[pos - 1, all_ids[pos]].item() for pos in range(prompt_len, len(all_ids)))
    return total, len(all_ids) - prompt_len


cases = [
    ("The capital of France is", [" Paris", " Lyon"]),
    ("The capital of Australia is", [" Canberra", " Sydney"]),
    ("The capital of Brazil is", [" Brasília", " Rio de Janeiro"]),
    ("The capital of the Kingdom of Zembla is", [" Onhava", " Paris"]),
]
for prompt, candidates in cases:
    top = log_probs_for(prompt)[-1]
    best = int(top.argmax())
    print(f"\n{prompt!r}   top-ranked next token: {tokeniser.convert_ids_to_tokens(best)!r} p = {math.exp(top[best].item()):.3f}")
    for cand in candidates:
        lp, n = score(prompt, cand)
        print(f"  {cand!r:<18s} {n} token{'s' if n > 1 else ' '}   ln p = {lp:8.3f}   p = {math.exp(lp):.6f}")

Output — what you should see

Loading weights: 100%|██████████| 311/311 [00:00<00:00, xxxx.xxit/s]

'The capital of France is'   top-ranked next token: 'ĠParis' p = 0.638
  ' Paris'           1 token    ln p =   -0.449   p = 0.638311
  ' Lyon'            1 token    ln p =   -5.011   p = 0.006661

'The capital of Australia is'   top-ranked next token: 'ĠSydney' p = 0.157
  ' Canberra'        1 token    ln p =   -3.352   p = 0.035031
  ' Sydney'          1 token    ln p =   -1.852   p = 0.156999

'The capital of Brazil is'   top-ranked next token: 'ĠBras' p = 0.256
  ' Brasília'        2 tokens   ln p =   -1.685   p = 0.185365
  ' Rio de Janeiro'  3 tokens   ln p =   -1.992   p = 0.136477

'The capital of the Kingdom of Zembla is'   top-ranked next token: 'Ġthe' p = 0.257
  ' Onhava'          3 tokens   ln p =  -22.541   p = 0.000000
  ' Paris'           1 token    ln p =  -10.483   p = 0.000028

Four readings, each of which returns later. Paris outscores Lyon by a factor of 96, so the 0.6B weights encode the capital of France in the sense just defined. For Australia the same checkpoint puts 0.157 on Sydney and 0.035 on Canberra: frequency, not truth, is what raised the probability. The likeliest explanation is corpus frequency, which cannot be checked because the corpus is not published; what can be measured is the ranking, and a widely repeated error is represented in the weights for the same reason a widely repeated fact is. Whether the 1.7B checkpoint ranks them the other way is a measurement you can make now: score.py defaults to the lab’s Qwen3-1.7B download, so run it with no argument once the download is done and record which of the two leads. Whichever way it goes, greedy decoding will print the leader as confidently as it printed Paris. For Brazil the two candidates have different token counts and the longer one pays for each token, which is worth remembering when Part 16’s harness lesson scores every option of a multiple-choice question by exactly this log-likelihood, without generating a word. And for the Kingdom of Zembla, which does not exist outside a novel, the model still produced a full distribution with Ġthe at 0.257: there is no boundary between what is in the weights and what is not, and nothing in the machinery reports “not found”. A third consequence follows from the first lesson of this page: the information has a date, because only the pretraining corpus shaped these numbers, and giving a model current documents at run time is Part 10’s subject.

Say “the weights encode” rather than “the model remembers”, and most of the confusion about these systems dissolves. The loss a fine-tune minimises in Part 13 is this same quantity computed on your examples and nothing else, which is where that lesson’s warning about forgetting comes from.

Probability of a sequence is not confidence in a fact

Consider two candidate answers to a question. One is a common fluent phrase; the other contains an unusual but correct identifier. The common phrase can receive greater model probability because the training objective rewards likely continuations. The objective does not consult a database of true propositions while sampling.

Sequence probability also multiplies conditional probabilities across positions. Longer sequences often have lower total probability simply because there are more factors. Comparing raw sequence probabilities across answers of different lengths is therefore a different operation from comparing their average token loss, and neither automatically measures factual correctness.

An application should separate generation from verification. For a product identifier, check membership in the permitted catalogue. For arithmetic, compute the result independently. For a document answer, inspect whether the cited passage supports the assertion. A model’s self-reported confidence is another generated string until calibrated against outcomes on representative held-out examples. In your notebook, distinguish token likelihood, task success and calibrated confidence rather than calling all three “accuracy”.

Recap

Every model in this course was trained on one objective: minimise the mean of -ln p for the true next token over every position of the training text, with the label supplied by the text itself. One forward pass predicts every position at once, and the model’s loss is exactly that arithmetic, 3.08 nats on the dry run’s sentence. Running a model produces one logit per vocabulary entry, which a softmax turns into a distribution 151,936 wide for the Qwen3 models. A decoding strategy picks one token: greedily, or by a seeded draw after temperature, top-k and top-p have reshaped the distribution, and q^n says how quickly sampled runs diverge. A chat assistant is this loop with a template that flattens roles and messages into one token sequence, five template tokens per message and an open assistant turn at the end, and the reply ends when the end-of-turn token is drawn. Confident false output follows from the objective, because the loss never saw the world, so anything that must be true is checked outside the model, and “the weights encode a fact” means a log-probability you can compute.

Check your understanding

Question 1. What does a language model produce when you run it on a prompt?

• The next word, as a string
• One logit per vocabulary entry, which a softmax turns into a probability distribution over the whole vocabulary
• A ranked list of the ten most likely sentences
• A confidence score between 0 and 1 for the whole answer

Show the answer and why

Answer: One logit per vocabulary entry, which a softmax turns into a probability distribution over the whole vocabulary

The logits tensor has the shape (batch, positions, vocab_size): 151,936 numbers per position for the Qwen3 models, computed by multiplying the last hidden state by the unembedding matrix. The decoding strategy chooses one token from the softmax of the last row; the rows for earlier positions are the predictions pretraining scored.

Question 2. You set temperature to 0.1 and the model still returns a wrong fact, now with fewer hedges. Why?

• Temperature is broken at low values
• Temperature reshapes a distribution the model already produced; if the wrong token had the highest probability, sharpening the distribution makes it more certain
• A low temperature disables the safety training
• The model needs a longer prompt

Show the answer and why

Answer: Temperature reshapes a distribution the model already produced; if the wrong token had the highest probability, sharpening the distribution makes it more certain

Dividing the logits by 0.1 raises every probability to the power 10 and renormalises, so the leader takes nearly everything. In the dry run, " Sydney" led " Canberra" after "The capital of Australia is"; at a low temperature that wrong leader is drawn almost every time. The fix is a check outside the model, not a sampler setting.

Question 3. Which statements about chat models are accurate? Select all that apply.

• A conversation is flattened into a single token sequence with control tokens marking the roles
• Chat models use a different objective from base models at pretraining time
• The assistant’s earlier replies are re-read as ordinary prompt tokens on every turn
• The reply ends when the model draws the end-of-turn token, which has a probability like any other

Show the answer and why

Answer: A conversation is flattened into a single token sequence with control tokens marking the roles, The assistant’s earlier replies are re-read as ordinary prompt tokens on every turn, The reply ends when the model draws the end-of-turn token, which has a probability like any other

The template adds five tokens per message, two of them special-token ids, and three for the open assistant turn; the model has no memory of its own replies beyond the tokens in the prompt, which is why prefill grows every turn. Pretraining uses the same next-token objective for both; what differs is the later post-training, which Part 3 covers.

Question 4. At every step of a 20-token answer the top-ranked token has probability 0.9. With sampling at temperature 1 and no truncation, what is the chance the sampled answer is identical to the greedy one?

• About 90 per cent, because each step is 90 per cent likely
• About 12 per cent, because 0.9 to the power 20 is 0.12
• Exactly 100 per cent, because the top token is always drawn
• It cannot be estimated without the seed

Show the answer and why

Answer: About 12 per cent, because 0.9 to the power 20 is 0.12

Each draw is independent, so the probabilities multiply: 0.9 to the power 20 is about 0.12, and at 100 tokens it is 0.00003. That is why two sampled runs of the same prompt almost never match, and why a measurement that must be repeatable uses greedy decoding or a fixed seed.

Question 5. A script builds a prompt for an instruct model with apply_chat_template(messages, add_generation_prompt=False, tokenize=True, return_dict=True) and then calls generate. The model answers by writing a second user message. Which line is the bug?

• tokenize=True: the template should be rendered as text and tokenised separately
• add_generation_prompt=False: nothing opened an assistant turn, so the likeliest continuation of a closed user turn is another user turn
• return_dict=True: generate needs a plain list of ids
• The call is correct; the model needs a system message

Show the answer and why

Answer: add_generation_prompt=False: nothing opened an assistant turn, so the likeliest continuation of a closed user turn is another user turn

add_generation_prompt=True appends the three tokens that open an assistant turn, <|im_start|>, assistant and a newline for Qwen3. Without them the sequence ends with <|im_end|> and a newline, and the documentation warns the model "may get confused and do something strange, like continuing the user's message instead of replying to it". tokenize=True is the form the documentation calls the safer option, because it cannot double the special tokens.

Question 6. Why is a fabricated but plausible citation a predictable consequence of the training objective?

• Because the training data was mostly fabricated
• Because the objective rewards continuations that look like the training text, and no step in the mechanism checks a claim against the world
• Because the tokeniser splits citations badly
• Because sampling was enabled

Show the answer and why

Answer: Because the objective rewards continuations that look like the training text, and no step in the mechanism checks a claim against the world

After "As shown in the 2019 study by" the dry run put half the probability on "the" and spread the rest over a field as uncertain as 83 equally likely tokens, with "Smith" among the top five. The loss only ever saw the next token in the corpus, never whether a study existed, so the shape of a citation is learned and the specifics are left to the draw. Verification has to come from retrieval, tools, tests or a person.

Check my answersClear