Skip to content
Level 3 · Model BuilderLessonPart 11 · page 4 of 630 min
30Minutes
7Sources

Datasets: Formats, Chat Templates, Tokenisation and Packing

By the end of this lesson you will be able to put a set of examples into the exact shape a trainer expects, apply the model’s own chat template to them, decide deliberately which tokens the loss is computed on, and build a validation split that can still tell you something. You will also know the four ways a dataset silently ruins a fine-tune, because none of them raise an error and all of them are visible before you start.

More fine-tunes are spoiled here than anywhere else in Level 3. A wrong learning rate produces a bad loss curve you can see. A wrong chat template produces a perfectly ordinary loss curve and a model that behaves oddly when served.

What happens to one example between your file and the optimiser

  1. A line of JSONOne example per line, loaded by datasets with load_dataset("json", data_files=...).
  2. A recognised shapeTRL reads a "text" field, or "messages", or a "prompt" and "completion" pair. Anything else has to be mapped into one of them.
  3. The chat templateMessages become a string with the model's own control tokens. Applied by the trainer for conversational data.
  4. Tokens and a label maskTokenised, truncated to max_length, and labelled: -100 marks every position excluded from the loss.
  5. A batchPadded, or packed with other examples, and handed to the model.
Five steps, each of which can be wrong without raising an error. The lesson takes them in order.

The course’s training data is JSON Lines: one JSON object per line, which the Datasets loading guide calls the most efficient JSON format and loads with load_dataset("json", data_files="my_file.json"). It is a good format for training data for reasons that have nothing to do with speed: a line is an example, wc -l is a count, head is a preview, and a diff of two versions is readable.

TRL’s dataset guide separates the format of an example from its type. The format is either standard, plain text strings, or conversational, lists of messages with role and content keys. The type is what the example is for. SFTTrainer accepts two of the types:

Type Standard format Conversational format
Language modelling {"text": "The sky is blue."} {"messages": [{"role": "user", "content": "What color is the sky?"}, {"role": "assistant", "content": "It is blue."}]}
Prompt-completion {"prompt": "The sky is", "completion": " blue."} {"prompt": [{"role": "user", ...}], "completion": [{"role": "assistant", ...}]}

The documentation states that SFTTrainer “is compatible with both standard and conversational dataset formats” and that “when provided with a conversational dataset, the trainer will automatically apply the chat template to the dataset”. Parts 14 and 15 need the other types, preference and prompt-only, and the same page lists which trainer takes which.

The choice between the two rows is not cosmetic, because it decides the default answer to the next question.

A causal language model is trained to predict the next token. Given a question and its answer concatenated into one sequence, you can compute the loss over every position, or only over the answer’s positions. TRL’s documentation is precise about the default: with a prompt-completion dataset “the trainer computes the loss on the completion tokens only, ignoring the prompt tokens”, controlled by completion_only_loss, and with a conversational dataset assistant_only_loss=True “ensures that loss is computed only on the assistant responses, ignoring user or system messages”.

Mechanically, masking is a label of -100. TRL describes the one-token shift and states that positions with the ignore index “are ignored in the loss computation”, which is the same mechanism padding uses.

The same example, three ways

Completion-only loss
prompt tokens, ignoredcompletion tokens, in the loss
Loss on everything
prompt tokens, in the losscompletion tokens, in the loss
Packed row
prompt Acompletion Aprompt Bcompletion Bpad
Shaded blocks contribute to the loss; pale blocks are labelled -100 and ignored. The third row is one packed training row holding two examples plus padding.

When does the choice matter? Train on everything and the model spends capacity learning to produce your questions as well as your answers, which is wasted at best and, on a narrow dataset of similar prompts, actively harmful. Train on completions only and every gradient is about the behaviour you want. For instruction tuning, completions only is nearly always right, and it is what the lab uses.

A chat model was fine-tuned on text with specific control tokens marking the turns. The Transformers guide puts it plainly: all causal language models continue a sequence of tokens, and the list of role and content dictionaries you pass “get converted to a token sequence, often with control tokens like <|user|> or <|assistant|> or <|end_of_message|>”. Different models use different ones, so each ships a chat template, applied with apply_chat_template.

Three rules follow, and each is a real failure mode.

Use the model’s own template, not one you like. The guide shows the same three-message chat rendered by two models fine-tuned from the same base into completely different token sequences, and states that “with the wrong control tokens, these models would have drastically worse performance”. AutoTokenizer.from_pretrained carries the right one. Where a base model has no template at all, SFTConfig takes chat_template_path to borrow one, and the lab’s script stops with a clear message rather than training against nothing.

Do not add generation prompts during training. add_generation_prompt=True appends the tokens that start an assistant turn, which is what you want when generating and not what you want when training. The guide’s own training section says to “set add_generation_prompt=False because the additional tokens to prompt an assistant response aren’t helpful during training”.

Do not tokenise twice. The guide warns that chat templates already include the necessary special tokens, and that adding more “is often incorrect or duplicated, hurting model performance”; if you format with tokenize=False and tokenise afterwards, pass add_special_tokens=False. Letting the trainer apply the template for you avoids the whole question.

Examples vary in length; a batch is rectangular. Three settings reconcile the two.

max_length truncates. TRL’s memory guide is blunt about the trade: “If max_length is too small, a significant portion of your tokens will be discarded and won’t contribute to training. If it’s too large, memory usage can spike”. Look at your data’s length distribution before choosing, because both failures are silent, one losing the ends of your answers and the other producing an out-of-memory error at whichever step the longest example lands in.

Padding fills the rest of each row. It costs memory and compute for tokens that are masked out of the loss anyway, which is what the other two settings attack.

Packing groups several examples into one row up to max_length. TRL implements best-fit decreasing bin packing and offers three strategies: "bfd", the default, which discards overflow; "bfd_split", which splits long sequences into chunks first so that no tokens are lost; and "wrapped", which concatenates everything into a stream and cuts fixed blocks, minimising padding but mixing unrelated examples. The documentation notes that when every sequence is shorter than max_length, bfd and bfd_split behave identically.

Part 1 taught the three-way split: train on one part, choose the checkpoint with the second, spend the third once. That lesson’s warnings apply here unchanged, and instruction data adds two of its own.

train_test_split(test_size=0.1) produces splits and shuffles them by default; shuffle(seed=...) takes a seed. Both need the seed recorded, or “the validation set” names a different set of examples on the next run and two loss curves stop being comparable.

Near-duplicates leak. Instruction datasets are full of examples that differ only in a name or a number. A random split puts one in train and its twin in validation, and the validation loss then measures recall rather than generalisation. Deduplicate before splitting, on normalised text rather than exact bytes.

Grouped data leaks through the group. If ten examples came from the same document, the same customer or the same template, split by that group and not by example. Part 13 returns to this when the dataset is one you built yourself and the groups are obvious only in hindsight.

Evaluation sets leak into training corpora. If you fine-tune on data scraped from the same place your benchmark came from, the benchmark stops measuring anything. Part 16 treats decontamination as a first-class step; here it is enough to know the failure exists and that a model scoring suspiciously well is a reason to look at the data rather than to celebrate.

Training data carries obligations, and they are easier to check before the run than to unpick afterwards. Three questions, every time.

What is the licence? A dataset on the Hub documents itself in a dataset card, which the Hub describes as the repository’s README.md with a YAML metadata block carrying fields including license, language, tags and task_categories. A dataset with no card and no licence field is a dataset you do not know the terms of. That is a reason to look elsewhere, not a reason to assume permission.

Where did it come from? The Hub’s guidance is that a card should help users “understand the contents of the dataset and give context for how the dataset should be used”, including potential biases. A dataset generated by a hosted model may carry terms from that model’s provider as well as from whoever published it. Part 15 raises this again for distillation, where the teacher’s terms are the whole question.

What does it contain? Personal data, credentials and copyrighted text can all be in a corpus that nobody meant to include them in. A model trained on them can reproduce them. Read a sample by hand; a hundred lines is fifteen minutes and it is the only step here that consistently finds things.

This part’s lab avoids all three questions by generating its dataset from a script in the course’s own repository, so the provenance is a file you can read and the licence is the course’s. That is the exception. In Part 13 you will build a dataset of your own, and these three questions are the first section of that page.

Inspect the exact learning signal on one example

Section titled “Inspect the exact learning signal on one example”

Take one conversation with a system instruction, user request and assistant answer. Render it with the training template, tokenise it and decode it again. Mark the assistant boundaries and the tokens selected by the loss mask. Padding positions should not accidentally teach the model to emit padding; a completion-only run should not silently train on the prompt.

Now use the longest example and apply the configured truncation. Confirm that the answer, closing delimiters and end-of-turn marker survive where required. A row that contains a valid target before tokenisation can become an unusable training example after truncation.

Finally inspect a packed batch containing two documents. Document delimiters and attention boundaries have different roles: an end-of-document token marks a boundary in the sequence, while masking or sequence-aware attention determines whether one example can attend to another. Check the packer’s actual implementation rather than inferring isolation from a separator token. Save this annotated batch with the dataset revision. It is often the quickest way to diagnose a fine-tune that learns the wrong conversational behaviour.

TRL accepts four shapes: standard or conversational, crossed with language modelling or prompt-completion, and applies the model’s chat template automatically to conversational data. The loss is computed on completion tokens only by default for prompt-completion data, and on assistant messages only when assistant_only_loss is set, which requires generation keywords in the template. Chat templates must be the model’s own, must not add a generation prompt during training, and must not be tokenised twice. max_length truncates, padding wastes, and packing recovers the waste where FlashAttention is available. Splits need a recorded seed, deduplication before splitting, grouping where the data is grouped, and decontamination against anything you intend to measure with. And every dataset that is not yours has a licence, a provenance and contents that are worth ten minutes before the run rather than an afternoon after it.

Check your understanding

Question 1. You have {"prompt": [...], "completion": [...]} examples and pass them to SFTTrainer without setting completion_only_loss. What is computed?
Show the answer and why

Answer: Loss on the completion tokens only, which is the documented default for prompt-completion datasets

TRL states that for prompt-completion datasets the trainer computes the loss on the completion tokens by default. Setting completion_only_loss=False is what asks for the whole sequence.

Question 2. Why does the course prefer the prompt-completion shape to assistant_only_loss for this part's lab?
Show the answer and why

Answer: assistant_only_loss requires the chat template to contain generation keywords, which TRL patches only for known model families

The requirement is documented, and TRL patches the template automatically for families such as Qwen3 but not for every model. The prompt-completion route gets the same masking with no template requirement, which makes it the safer default to teach.

Question 3. Which of these leak between a training set and a validation set? Select all that apply.
Show the answer and why

Answer: Near-duplicate examples that differ only in a name or a number, Ten examples drawn from one source document, split randomly, Fine-tuning on text scraped from the source of your benchmark

A fixed seed is the opposite of a leak: it makes the split reproducible. The other three all let a model score well by recall, which is precisely what a validation set exists to rule out.

Question 4. A fine-tune evaluates well but answers strangely when served through llama.cpp. What is the first thing to check?
Show the answer and why

Answer: Whether the chat template used at serving time is the one the model was trained through

Evaluation loss is computed inside the training stack, using its template. Serving may apply a different one, and the model then sees control tokens it was never trained on. Recording the template, or its hash, in the run log makes this a one-minute check.

Question 5. Your GPU runs out of memory at step 400 of a run that started comfortably. Which explanation fits best?
Show the answer and why

Answer: The batch containing the dataset's longest examples arrived, and activation memory scales with the actual sequence length

Optimiser states are fixed by the parameter count and do not grow during a run. Activations scale with the sequence length actually being processed, so a max_length chosen against the average will fail against the maximum.

Sources for this lesson

7 verified · checked 2026-09-09

  1. 01TRL documentation — Dataset formats and types§ Overview; Standard; Conversational; Prompt-completion; Which dataset type to usehuggingface.co/docs/trl/main/en/dataset_formats2026-09-09
  2. 02TRL documentation — SFT Trainer§ Expected dataset type and format; Train on completion only; Train on assistant messages only; Packinghuggingface.co/docs/trl/main/en/sft_trainer2026-09-09
  3. 03TRL documentation — Reducing memory usage§ Truncation; Packing; Padding-freehuggingface.co/docs/trl/main/en/reducing_memory_usage2026-09-09
  4. 04Transformers documentation — Chat templates§ Using apply_chat_template; add_generation_prompt; Model traininghuggingface.co/docs/transformers/main/en/chat_templating2026-09-09
  5. 05Datasets documentation — Load§ JSON; Local and remote fileshuggingface.co/docs/datasets/main/en/loading2026-09-09
  6. 06Datasets documentation — Process§ Split; Shuffle; Maphuggingface.co/docs/datasets/main/en/process2026-09-09
  7. 07Hugging Face Hub documentation — Dataset cards§ Dataset card metadatahuggingface.co/docs/hub/datasets-cards2026-09-09

Every technical claim on this page was checked against the official documentation of the tool, vendor or model publisher on the date shown, at the version pinned for the course. Where the course disagrees with folklore, the source is how you can tell which one to trust.