What Learning Means: Data, Loss and Gradient Descent

Here is the sentence this part is built on. A model is a function with adjustable numbers in it, and learning is the process of adjusting those numbers until the function’s outputs match the examples you gave it. Everything else in machine learning is detail on top of that sentence: what the function looks like, how you measure “match”, how you adjust, and how you know whether the matching will hold for examples you did not give it. By the end of this lesson you will be able to run the adjusting loop by hand on one parameter and check every number against a one-line formula, predict from that formula when a learning rate is too large, read “loss 2.31” in a training log as a probability, and draw the loop that every training run in this course is an instance of. Every number in the tables and snippets is arithmetic from stated inputs, not a measurement; the few loss values quoted from this part’s lab are that page’s own recorded output. Every snippet but the last runs with a bare python3.

Learning is fitting a function

Take the smallest case that has all the parts: four examples of an input x with the output y that went with it, and a rule with one adjustable number, prediction = w * x. The number w is the rule’s parameter; the rule’s shape, a line through the origin, is the model; the four examples are the training data; and training is choosing w from the examples rather than by hand. This is supervised learning: every training example carries the answer y that the loss will compare the prediction against. Next-token prediction has exactly this form, since a text supplies its own answers, the token that actually came next; and Part 13’s supervised fine-tuning is the same loop run with your examples as the answers.

x	y
1	2.2
2	3.9
3	6.1
4	8.0

Nothing changes when the rule has eight billion parameters instead of one, or when the input is a sequence of tokens and the output is a probability for every entry in a vocabulary. A language model is a function of exactly this kind, with its parameters found by the procedure below run on a great deal of text; Part 2’s next-token prediction lesson looks at its output.

The loss: a number that says how wrong you are

To choose between candidate values of w you need one number per candidate that says how badly it fits the examples. That is the loss function: zero when prediction and answer agree, larger the further apart they are, averaged over the examples. Training is the search for the parameters that make it smallest. For a numerical output the usual loss is the squared error:

loss(w) = mean over examples of (w * x - y)^2

candidate w	predictions for x = 1, 2, 3, 4	errors (prediction − y)	mean squared error
0	0, 0, 0, 0	−2.2, −3.9, −6.1, −8.0	30.315
1	1, 2, 3, 4	−1.2, −1.9, −3.1, −4.0	7.665
2	2, 4, 6, 8	−0.2, +0.1, −0.1, 0.0	0.015
2.01	2.01, 4.02, 6.03, 8.04	−0.19, +0.12, −0.07, +0.04	0.01425
3	3, 6, 9, 12	+0.8, +2.1, +2.9, +4.0	7.365

Notice the floor: the four points lie on no line through the origin, so 0.01425 is the smallest this rule can reach on them, and that fact returns at every scale.

Cross-entropy, the loss of every language model

When the output is a choice among categories, the model outputs a probability per category and the loss charges for the probability it failed to give the correct one:

cross_entropy = -ln(p_correct)

probability given to the correct class	loss = −ln p
0.99	0.0101
0.9	0.1054
0.5	0.6931
0.25	1.3863
0.1	2.3026
0.01	4.6052
0.001	6.9078

Halving the probability costs the same 0.693 wherever you start, which puts confident and unconfident tokens on one scale and makes the mean over a batch meaningful.

Networks output one unbounded score per class, a logit; softmax turns scores into probabilities by exponentiating each and dividing by the sum. PyTorch’s nn.CrossEntropyLoss does both steps: its documentation says the input “is expected to contain the unnormalized logits for each class”, and its default reduction of 'mean' averages over the batch. Four logits, worked:

logit	exp(logit)	probability = exp ÷ 11.5804	loss if this class is correct
2.0	7.3891	0.6381	0.4493
1.0	2.7183	0.2347	1.4493
0.1	1.1052	0.0954	2.3493
−1.0	0.3679	0.0318	3.4493

Predicting the next token is a choice among vocabulary entries, so cross-entropy is the loss every language model in this course is trained with, and “loss 2.31” in a log is its mean over a batch. Two conversions make the number readable. Outputs spread evenly over C classes give each 1/C and pay ln(C): the starting line an untrained network prints and a broken one gets stuck at. And exp(loss) is the number of equally likely options the model is choosing among, its perplexity, which Part 16’s quantisation-damage lesson measures models with.

classes C	untrained loss ln(C)	where you will see it
2	0.6931	any yes/no classifier
10	2.3026	this part’s lab: its epoch 0 (untrained) line prints val loss 2.3091, ln(10) plus what the random initial weights happen to prefer
32,000	10.3735	a language model with a 32,000-entry vocabulary, at step 0
151,936	11.9312	Qwen3-8B (Apache-2.0), whose config.json gives vocab_size 151936: a freshly initialised model of that shape starts near 11.9

logged loss	exp(loss)	read as
2.31	10.07	as uncertain as a fair choice among ten
0.87	2.39	between two and three live options per token
0.19	1.21	the lab’s digit classifier after five epochs: nearly sure, nearly always

∴Research note

The loss measures the training examples, not the world. Whether the examples stand in for the cases you care about is the question the generalisation lesson answers. Keep “how well does it fit what I showed it” and “how well will it do on what I did not” separate.

The gradient: which way is downhill

With one parameter the loss is a curve over w, the parabola in the table above, lowest near 2.01. With two it is a surface; with billions it is something you cannot picture but can treat identically. You are standing somewhere on it and want to get lower.

The gradient answers, for each parameter, “if I increase this one a little, does the loss go up or down, and how steeply?” It is the slope of the loss with respect to each parameter, and for the squared-error loss it has a closed form you can check with a pencil:

gradient(w) = d loss / d w = 2 * mean over examples of ((w * x - y) * x)

Gradient descent is then one rule, applied to every parameter at once:

w_next = w - learning_rate * gradient(w)

which is the line the torch.optim.SGD documentation writes as θ_t ← θ_{t-1} − γ g_t. Step against the slope and the loss falls; repeat. You never need a map of the whole landscape, only the slope under your feet, and calculus provides that slope for any loss you can write down.

RunnableAll tracks

fit-one-parameter.py - gradient descent on one parameter, every number printed

"""Gradient descent on one parameter: four examples, y_hat = w * x, squared-error loss,
the slope computed by hand. Arithmetic, not a measurement."""
xs = [1.0, 2.0, 3.0, 4.0]
ys = [2.2, 3.9, 6.1, 8.0]

def loss(w):                    # mean squared error over the four examples
    return sum((w * x - y) ** 2 for x, y in zip(xs, ys)) / len(xs)

def gradient(w):                # d loss / d w = 2 * mean((w*x - y) * x)
    return 2 * sum((w * x - y) * x for x, y in zip(xs, ys)) / len(xs)

lr = 0.05
w = 0.0
print(f"{'step':>4} {'w':>8} {'loss':>9} {'gradient':>10} {'next w':>8}")
for step in range(10):
    g = gradient(w)
    w_next = w - lr * g         # the update rule: step against the slope
    print(f"{step:>4} {w:>8.4f} {loss(w):>9.4f} {g:>10.4f} {w_next:>8.4f}")
    w = w_next

m2 = sum(x * x for x in xs) / len(xs)
mxy = sum(x * y for x, y in zip(xs, ys)) / len(xs)
print(f"\nclosed-form best w = mean(x*y)/mean(x*x) = {mxy:.4f}/{m2:.4f} = {mxy / m2:.4f}")
print(f"learning-rate ceiling for this loss = 1/mean(x*x) = {1 / m2:.4f}")

Output — what you should see

step        w      loss   gradient   next w
   0   0.0000   30.3150   -30.1500   1.5075
   1   1.5075    1.9080    -7.5375   1.8844
   2   1.8844    0.1326    -1.8844   1.9786
   3   1.9786    0.0216    -0.4711   2.0021
   4   2.0021    0.0147    -0.1178   2.0080
   5   2.0080    0.0143    -0.0294   2.0095
   6   2.0095    0.0143    -0.0074   2.0099
   7   2.0099    0.0143    -0.0018   2.0100
   8   2.0100    0.0143    -0.0005   2.0100
   9   2.0100    0.0143    -0.0001   2.0100

closed-form best w = mean(x*y)/mean(x*x) = 15.0750/7.5000 = 2.0100
learning-rate ceiling for this loss = 1/mean(x*x) = 0.1333

Read the first row against the formula: at w = 0 the errors are the four y values, so 2 * (-2.2 - 7.8 - 18.3 - 32.0) / 4 = -30.15; the slope is negative, so the step goes up, 0 - 0.05 * (-30.15) = 1.5075. Nobody shrank the later steps: the gradient itself shrinks as w nears the bottom, from −30 to −0.0001, so a fixed learning rate slows down by itself, and after ten steps the loss sits on the 0.0143 floor. This loss has a closed-form answer and the loop finds it; a network’s loss has no such formula, which is the whole reason for the loop, and the next lesson shows how backpropagation produces the slope for every parameter of a deep network in one pass.

Momentum and Adam are still gradient descent

The other optimiser names in training configurations change how big and how steady the steps are, not what the steps follow. With g the gradient of one parameter and lr the learning rate, the PyTorch 2.14 documentation gives:

Optimiser	The step for one parameter	Extra numbers per parameter	Where it returns
Plain SGD (torch.optim.SGD, momentum=0)	w ← w − lr·g	none	this part’s lab, --lr 0.1
SGD with momentum	b ← μ·b + g, then w ← w − lr·b: the buffer remembers past gradients, so consistent directions accelerate and alternating ones cancel	one (b)	rarely in this course
AdamW (torch.optim.AdamW)	m ← β₁·m + (1−β₁)·g and v ← β₂·v + (1−β₂)·g², running means of the gradient and of its square; then w ← w − lr·m̂ / (√v̂ + ε), so each parameter’s step is scaled by how noisy its own gradient has been; weight decay is applied to w separately	two (m, v)	every transformer training configuration from Part 11 on; the two copies are the “optimiser states” in Part 11’s memory arithmetic

AdamW’s documented defaults are lr=0.001, betas=(0.9, 0.999), eps=1e-08 and weight_decay=0.01. The lab uses plain SGD so that the update is one multiplication you can see; everything with a transformer in it uses AdamW and pays two extra copies of every parameter for it.

The knobs: learning rate, batch size, epochs

The PyTorch tutorial defines the three hyperparameters as the number of times to iterate over the dataset, the number of samples propagated through the network before the parameters are updated, and how much to update the parameters at each step. Here is what each does to the arithmetic.

Learning rate: the ceiling and the crawl

On the one-parameter parabola the learning rate’s effect is exact. Each step multiplies the distance from w to the bottom by |1 − 2·lr·mean(x²)|: below 1/mean(x²), which is 0.1333 for these examples, the distance shrinks every step; above it, it grows every step, and the loss grows with it.

RunnableAll tracks

learning-rate-sweep.py - the same problem under seven learning rates

"""The one-parameter problem under seven learning rates, twenty steps each. Each step
multiplies the distance to the bottom by |1 - 2*lr*mean(x*x)|; the ceiling is
1/mean(x*x) = 0.1333. Arithmetic, not a measurement."""
xs = [1.0, 2.0, 3.0, 4.0]
ys = [2.2, 3.9, 6.1, 8.0]

def loss(w):
    return sum((w * x - y) ** 2 for x, y in zip(xs, ys)) / len(xs)

def gradient(w):
    return 2 * sum((w * x - y) * x for x, y in zip(xs, ys)) / len(xs)

print(f"{'lr':>6} {'w step 1':>9} {'w step 2':>9} {'w step 20':>10} {'loss step 20':>13}  verdict")
for lr in [0.001, 0.01, 0.05, 0.10, 0.13, 0.14, 0.20]:
    w, trace = 0.0, []
    for _ in range(20):
        w -= lr * gradient(w)
        trace.append(w)
    final = loss(w)
    verdict = "diverged" if final > loss(0.0) else "converging"
    print(f"{lr:>6.3f} {trace[0]:>9.3f} {trace[1]:>9.3f} {trace[19]:>10.3f} {final:>13.4g}  {verdict}")

Output — what you should see

    lr  w step 1  w step 2  w step 20  loss step 20  verdict
 0.001     0.030     0.060      0.524         16.57  converging
 0.010     0.301     0.558      1.932       0.05977  converging
 0.050     1.508     1.884      2.010       0.01425  converging
 0.100     3.015     1.507      2.010       0.01425  converging
 0.130     3.919     0.196      1.289         3.908  converging
 0.140     4.221    -0.422    -11.512          1371  diverged
 0.200     6.030    -6.030 -2107635.750     3.332e+13  diverged

Four regimes, and you will meet all four in training logs for the rest of the course:

What the loss does	What is happening	First move
Creeps down, nowhere near the floor at the end (0.001: 16.57 after twenty steps)	Far below what the surface allows	Multiply by 10; if the loss still does not move, suspect the data pipeline
Falls smoothly to the floor and stays (0.05, 0.10)	Working; the floor belongs to the rule and the data	Change the model or the data, not the rate
Overshoots and comes back from the other side (0.13 swings between 3.9 and 0.2, still 1.29 after twenty)	Just under the ceiling: converging by oscillation	Lower it a little, or let a decay schedule do it
Grows from the first steps to absurd values or nan (0.14, 0.20)	Above the ceiling: every step lands further from the bottom	Divide by 10

The ceiling is 1/mean(x²), so it depends on the scale of the inputs: double every x and the safe rate falls by four, one reason inputs are normalised and, in the next lesson, why layers are. A network has no closed-form ceiling but it has one; the lab’s Task 8 shows both ends, a first-epoch training loss in the hundreds at --lr 20 and a crawl from 2.31 to 2.29 in three epochs at --lr 0.0001.

Schedules exist because the best rate is not constant. Warmup raises it from near zero over the first steps, while random initial weights make the gradient’s direction untrustworthy; a warm-down (linear or cosine) lowers it towards the end so the final steps settle into the bottom instead of bouncing across it, as the 0.13 row did. Part 12’s anatomy of a training loop reads nanochat’s warm-up, constant and linear warm-down schedule line by line.

✕Common mistake

A loss that rises from the first step is not “the model needs more time”. It is the last row of the table: the rate is above the ceiling, and more steps make it worse. The symptom is a first-epoch training loss far above ln(C), then nan. Divide the rate by ten before touching anything else.

Batch size: noise against cost

The batch size is how many examples the gradient is computed from before a step is taken. The whole training set gives the exact gradient at the cost of a full pass per step; one example gives a cheap gradient that is mostly noise. The noise is measurable: compute the exact gradient over 50,000 synthetic examples, then from random batches, and see how much the batch gradient spreads.

RunnableAll tracks

batch-noise.py - how noisy a batch gradient is, by batch size

"""50,000 examples of y = 2*x + noise. The exact gradient at w = 1.5 uses all of them;
a batch gradient uses B. The spread across 200 random batches is the noise gradient
descent lives with. Synthetic arithmetic, not a measurement."""
import math
import random
import statistics

random.seed(0)
N = 50_000
xs = [random.uniform(0.0, 4.0) for _ in range(N)]
ys = [2.0 * x + random.gauss(0.0, 1.0) for x in xs]
w = 1.5

def gradient(indices):          # d loss / d w over the chosen examples
    return 2 * sum((w * xs[i] - ys[i]) * xs[i] for i in indices) / len(indices)

exact = gradient(range(N))
print(f"exact gradient at w = {w} over all {N:,} examples: {exact:.4f}\n")
print(f"{'batch B':>8} {'steps/epoch':>12} {'gradient std':>13} {'std x sqrt(B)':>14}")
for B in [1, 8, 64, 512, 4096]:
    samples = [gradient(random.sample(range(N), B)) for _ in range(200)]
    std = statistics.stdev(samples)
    print(f"{B:>8} {math.ceil(N / B):>12,} {std:>13.4f} {std * math.sqrt(B):>14.3f}")

Output — what you should see

exact gradient at w = 1.5 over all 50,000 examples: -5.3420

 batch B  steps/epoch  gradient std  std x sqrt(B)
       1       50,000        6.6290          6.629
       8        6,250        2.2124          6.258
      64          782        0.8228          6.583
     512           98        0.2942          6.657
    4096           13        0.1006          6.441

The last column is the point: it is flat. The spread of a batch gradient falls as 1/√B, so quadrupling the batch halves the noise at four times the arithmetic per step; a batch of one gives a spread (6.6) larger than the gradient itself (−5.3), a batch of 512 pins it to a few percent. Because every example’s activations are held until the backward pass, batch size is also a memory decision, the term Part 11’s memory arithmetic computes; too large a batch fails at the first step with the accelerator’s out-of-memory error, and the fix is to halve the batch and accumulate gradients, as follows: when the batch that fits is smaller than the batch you want, the Part 11 trainers add up the gradients of several small batches before stepping (gradient accumulation), buying the noise of the larger batch at the memory cost of the smaller.

Epochs, steps and tokens

An epoch is one pass through the whole training set; a step is one execution of the loop, one batch. The bookkeeping for this part’s lab, with N training examples and batch B:

Quantity	Formula	The lab’s default run
Steps per epoch	ceil(N / B)	ceil(50,000 / 128) = 391, the 391 steps per epoch the script prints
Examples in the last batch	N − (steps − 1) × B	50,000 − 390 × 128 = 80
Steps in the run	epochs × steps per epoch	5 × 391 = 1,955
Gradient noise relative to the default batch	√(128 / B)	B = 512: half the noise, a quarter of the steps

Language models are counted in steps and tokens rather than epochs; the bookkeeping is the same with N in tokens, and the relation between tokens, parameters and loss is Part 12’s scaling laws.

The loop

Put the pieces together and the whole of training is this:

The training loop

1. 1BatchTake the next batch of examples.
2. 2ForwardRun them through the model to get predictions.
3. 3LossCompare predictions with the answers: one number.
4. 4BackwardZero the old gradients, then compute the gradient of the loss for every parameter.
5. 5StepMove every parameter against its gradient, scaled by the learning rate.

Every training run in this course, from the digit classifier to a fine-tune of an eight-billion-parameter model, is this loop. Later lessons change what sits inside each box; none changes the boxes.

Repeat for every batch, for as many epochs as you chose, watching the loss. The lab’s train-mnist.py writes the loop in four marked lines, and every later part keeps the boxes and changes their contents:

Box	train-mnist.py (this part’s lab)	Pretraining (Part 12)	Fine-tuning (Part 13)	Preferences and RL (Part 14)
Batch	128 images and their labels	A block of tokens; the target at each position is the next token	Chat examples rendered through a template	A preferred and a rejected answer to one prompt (DPO); one prompt and several sampled answers (GRPO)
Forward	logits = model(images)	The transformer	The transformer, often through LoRA adapters	The model, plus a frozen reference copy (DPO)
Loss	loss = loss_fn(logits, labels): cross-entropy over ten classes	Cross-entropy over the vocabulary at every position	The same, counted on the answer tokens only	From the log-probabilities of the two answers (DPO); from a reward per sampled answer (GRPO)
Backward	loss.backward()	The same	The same, for the adapter parameters only	The same
Step	optimiser.step(): plain SGD	AdamW on embeddings and scalars, Muon on the matrices; linear warm-up, constant middle, linear warm-down	AdamW; the TRL lesson sets learning_rate 2e-5	AdamW; the rate is set per method in Part 14

Distillation (Part 15) changes no box: the teacher writes the examples and the student runs the Part 13 column on them. Every framework hides this loop behind a call, but it is always there, and when a run misbehaves it is the loop you reason about.

One line in it is easy to leave out, and its failure is silent. The Tensor.backward documentation says it “accumulates gradients in the leaves - you might need to zero .grad attributes or set them to None before calling it”. Here is the hand-worked step reproduced by autograd and SGD, then the bug:

RunnableAll tracks

same-step-in-torch.py - the hand-computed step, reproduced by autograd and SGD

"""The hand-computed gradient and step, reproduced by PyTorch's autograd and SGD.
Needs the environment built in the lab at the end of this part (torch installed)."""
import torch

xs = torch.tensor([1.0, 2.0, 3.0, 4.0])
ys = torch.tensor([2.2, 3.9, 6.1, 8.0])
w = torch.tensor(0.0, requires_grad=True)    # one parameter, starting at 0
optimiser = torch.optim.SGD([w], lr=0.05)

loss = ((w * xs - ys) ** 2).mean()           # forward + loss
loss.backward()                              # backward: fills w.grad
print(f"loss {loss.item():.4f}  w.grad {w.grad.item():.4f}   (by hand: 30.3150 and -30.1500)")
optimiser.step()                             # w <- w - lr * w.grad
print(f"after one step: w = {w.item():.4f}   (by hand: 1.5075)")

# The bug the tutorial warns about: gradients add up unless you zero them.
loss = ((w * xs - ys) ** 2).mean()
loss.backward()
print(f"second backward without zero_grad(): w.grad = {w.grad.item():.4f}  (fresh gradient would be -7.5375)")
optimiser.zero_grad()
loss = ((w * xs - ys) ** 2).mean()
loss.backward()
print(f"second backward after zero_grad():   w.grad = {w.grad.item():.4f}")

Output — what you should see

loss 30.3150  w.grad -30.1500   (by hand: 30.3150 and -30.1500)
after one step: w = 1.5075   (by hand: 1.5075)
second backward without zero_grad(): w.grad = -37.6875  (fresh gradient would be -7.5375)
second backward after zero_grad():   w.grad = -7.5375

Produced with torch 2.14.0, the version the lab installs. Without zero_grad() the second gradient is -30.15 + -7.5375 = -37.6875, the sum of both; a loop that forgets the call takes steps that grow with every batch and diverges as surely as a rate above the ceiling, which is why the lab’s script calls optimiser.zero_grad() immediately before loss.backward().

Why more data beats cleverer rules

One last idea, because it explains the shape of the whole field, and it too can be shown on the four examples. The one-parameter line could not reach zero loss on them; its floor was 0.01425. A cleverer rule can. A cubic, a·x³ + b·x² + c·x + d, has four parameters, and four parameters can be chosen to pass exactly through four points:

Rule	Parameters	Loss on the four examples	Prediction at x = 5	Prediction at x = 6
Line through the origin, 2.01·x	1	0.01425	10.05	12.06
Cubic, −0.1333·x³ + 1.05·x² − 0.5167·x + 1.8	4	0.0	8.8	7.7

The cubic has the better loss and has learned nothing: every example showed y growing with x, and the cubic has already stopped rising at x = 5 (its slope there is −0.02) and predicts 7.7 at x = 6, below its own value at x = 4, because with as many parameters as examples the loss reaches zero by fitting the noise in the four y values rather than the pattern behind them. With one parameter the only way to lower the loss was to find the slope, and that is the mechanism in general: when examples greatly outnumber parameters, the only way to get them all right at once is to capture what they share.

The second half of the mechanism is that noise averages out. Fit the one-parameter line to n noisy examples of y = 2x + noise and measure how far the learned w lands from 2:

RunnableAll tracks

more-data.py - how close the learned parameter gets as the examples grow

"""Examples from y = 2*x + noise; the fitted w is what gradient descent converges to
(its closed form). 500 repeats per size so the mean error is stable. Synthetic
arithmetic, not a measurement."""
import random

random.seed(0)
TRUE_W, REPEATS = 2.0, 500

def fit(n):
    xs = [random.uniform(0.0, 4.0) for _ in range(n)]
    ys = [TRUE_W * x + random.gauss(0.0, 1.0) for x in xs]
    return sum(x * y for x, y in zip(xs, ys)) / sum(x * x for x in xs)

print(f"{'examples n':>10} {'mean |w - 2|':>13} {'error x sqrt(n)':>16}")
for n in [4, 16, 64, 256, 1024, 4096]:
    err = sum(abs(fit(n) - TRUE_W) for _ in range(REPEATS)) / REPEATS
    print(f"{n:>10} {err:>13.4f} {err * n ** 0.5:>16.3f}")

Output — what you should see

examples n  mean |w - 2|  error x sqrt(n)
         4        0.1883            0.377
        16        0.0893            0.357
        64        0.0435            0.348
       256        0.0224            0.359
      1024        0.0107            0.343
      4096        0.0053            0.342

Again the last column is flat: the error in the learned parameter falls as 1/√n, so every quadrupling of the data halves the distance to the pattern with no change to the rule. A cleverer rule, a better architecture or a smarter optimiser, moves the floor once; more examples keep paying. That is why language models are trained on as much text as can be gathered, why the fine-tuning lessons spend more time on the dataset than on the trainer (Part 13’s what fine-tuning changes says how much data and how good), and why distillation is mostly about generating good examples. Where to spend effort, then, using the training-versus-held-out comparison the generalisation lesson teaches:

What you measured	Where to spend the effort	Why
Training loss high and flat, held-out loss the same	The learning rate first, then the model’s capacity	The rule cannot fit even what it has seen: the floor is the rule’s
Training loss near zero, held-out loss high	More examples, or a simpler rule	The cubic case: zero loss by memorising, because n is not much larger than the parameter count
Both losses still falling together at the end of the run	More steps, then more data	The run has not finished; the 1/√n curve is still paying
Held-out loss at a floor, and n already large	A different rule	Only now is the floor the architecture’s

Choosing a learning problem before choosing a model

Suppose a support team wants to route tickets. Write the input as the text available when the ticket arrives, and the target as the team that should handle it. A resolution note written later is unavailable at prediction time; including it would make the training exercise easier while making the deployed system unusable. The unit of prediction and the time at which information becomes available are part of the problem definition.

Start with a baseline: route every ticket to the most frequent team, then compare a keyword rule or a simple classifier. A language model earns its extra computation only if it improves the decision you care about. Accuracy alone can hide a failure on rare urgent tickets; count those mistakes separately and decide who handles uncertain cases.

This is where LLM work meets AI more broadly. The AI problem-solving map distinguishes classification, regression, clustering, retrieval, generation and sequential decisions. The learning loop here applies to many of them, but their targets, losses and acceptance tests differ. Before the lab, write down your input, target, baseline and one costly error in ordinary language.

Recap

A model is a function with parameters; training searches for the parameters that make a loss small over the training examples: squared error for a quantity, cross-entropy -ln(p_correct) for a choice, which starts at ln(C) and reads as a perplexity of exp(loss). Gradient descent steps against the slope, w ← w − lr·gradient; momentum and AdamW change the step’s size and steadiness for one or two extra numbers per parameter. The learning rate has a ceiling; batch noise falls as 1/√B; steps per epoch are ceil(N/B). The loop is batch, forward, loss, backward, step, gradients zeroed before each backward pass, at every scale in this course. More data beats a cleverer rule because a rule with as many parameters as examples memorises, while a fixed rule’s error falls as 1/√n.

Check your understanding

Question 1. A training log prints "loss 2.31" for a language model. What is that number?

• The number of tokens the model got wrong in the batch
• The mean cross-entropy over the batch: -ln of the probability the model gave the correct next token, averaged
• The learning rate after warmup
• The percentage of the dataset processed so far

Show the answer and why

Answer: The mean cross-entropy over the batch: -ln of the probability the model gave the correct next token, averaged

Predicting the next token is a choice among categories, so the loss is cross-entropy averaged over the batch. exp(2.31) is 10.07: as uncertain as a fair choice among ten options per token, which for a vocabulary of over a hundred thousand entries means the model has already learned a great deal; untrained, it would print ln(151936) = 11.93.

Question 2. On one token the probability the model gives the correct answer drops from 0.9 to 0.45; on another it drops from 0.10 to 0.05. Which token adds more to the loss?

• The first, because 0.45 is a much larger absolute drop than 0.05
• The second, because the model was already nearly wrong
• Both add the same amount, ln 2 = 0.693
• Neither changes the loss; only the most probable class matters

Show the answer and why

Answer: Both add the same amount, ln 2 = 0.693

Cross-entropy is -ln p, so halving the probability costs ln 2 = 0.693 wherever you start: 0.1054 to 0.7985 on the first token, 2.3026 to 2.9957 on the second. The logarithm puts confident and unconfident tokens on one scale.

Question 3. Which line makes this training step wrong?

for images, labels in train_loader:
    logits = model(images)
    loss = loss_fn(logits, labels)
    loss.backward()
    optimiser.step()

• The loss is computed before the backward pass; it must come after
• There is no optimiser.zero_grad() before loss.backward(), so each batch's gradient is added to the previous ones and the steps grow until the run diverges
• optimiser.step() must be called before loss.backward()
• Nothing; this is the loop from the lesson

Show the answer and why

Answer: There is no optimiser.zero_grad() before loss.backward(), so each batch's gradient is added to the previous ones and the steps grow until the run diverges

backward() accumulates into .grad by default: on the one-parameter problem the second gradient without zeroing was -37.69 instead of -7.54, the sum of both. The order forward, loss, backward, step is right; the missing zero_grad() is the bug, and its symptom looks like a learning rate that is too large.

Question 4. On the four-example problem, learning rate 0.10 converges and 0.14 diverges, because the ceiling is 1/mean(x²) = 0.1333. You double every input, so x becomes 2, 4, 6, 8. What is the ceiling now?

• Still 0.1333; the ceiling depends only on the loss function
• 0.0667, half
• 0.0333, a quarter
• 0.2667, double

Show the answer and why

Answer: 0.0333, a quarter

mean(x²) goes from 7.5 to 30, so the ceiling falls from 0.1333 to 0.0333. The safe learning rate depends on the scale of what flows into the parameters, which is why inputs are normalised and why the next lesson's networks normalise between layers.

Question 5. You raise the batch size from 128 to 512 and keep the number of epochs. What happens to the steps per epoch and to the noise in each gradient?

• Steps per epoch fall from 391 to 98; gradient noise halves
• Steps per epoch fall from 391 to 98; gradient noise falls to a quarter
• Steps per epoch are unchanged; gradient noise halves
• Steps per epoch fall from 391 to 98; gradient noise is unchanged

Show the answer and why

Answer: Steps per epoch fall from 391 to 98; gradient noise halves

Steps per epoch are ceil(50,000 / 512) = 98. Gradient noise falls as one over the square root of the batch size, so four times the batch gives half the noise, at four times the arithmetic and memory per step. The run also takes a quarter as many steps, so at the same rate it moves the parameters less in total, so the rate is normally re-tuned when the batch changes.

Check my answersClear