Base, Instruct, Thinking, Coder, Vision, Embedding: Reading a Model Name

By the end of this lesson you will be able to take any repository identifier apart, confirm each field from the repository’s own files in under a minute, and choose the variant a job needs: base or post-trained, which of four thinking designs and how many tokens to budget for it, a coder that really does fill in the middle, a vision-language model and its image token bill, an embedding model or a reranker, and a distilled model, a quantisation-aware one or a plain re-quantisation.

A model name is a claim written by whoever created the repository, in a convention nobody documents in one place. Every field in it has a counterpart in the files and metadata, which are harder to fudge than a name: config.json, generation_config.json, the chat template, the tokeniser, and the metadata the Hub derives from them. This lesson reads the name and those files side by side.

The anatomy of an identifier

Take a fully loaded identifier, a republication of Qwen3-Coder-30B-A3B-Instruct (Apache-2.0), in the form llama-server --hf-repo accepts, which the llama.cpp v0.4.0 · verified 2026-09-08 README documents as <user>/<model>[:quant], with the quantisation “optional, case-insensitive, default to Q4_K_M”:

Pseudocode — not a real command

unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M

unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M, taken apart

1. unslothThe namespace that created these files. Here it is a republisher: the Hub tags this repository as quantized from Qwen/Qwen3-Coder-30B-A3B-Instruct. The card and licence you read are that namespace's.
2. Qwen3-CoderFamily, generation and specialisation. Sets the architecture class, the tokeniser and the chat template, which is what engine support depends on.
3. 30BTotal parameters, rounded (30.5 billion on the card). Sets the memory the weights need.
4. A3BActive parameters per token, on mixture-of-experts models (3.3 billion). Sets how many bytes each decoded token reads, and so decode speed.
5. InstructThe post-training variant, the field this lesson is mostly about. Some families put the role in the family field instead: Coder, VL, Embedding, Reranker.
6. GGUFThe file format, when it is not safetensors. Decides which engines can load it.
7. Q4_K_MThe quantisation tier, which selects one file: Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf, 18,556,689,568 bytes in the Hub listing on 2026-09-12.

Not every name carries every field, and the defaults for a missing field are habits rather than rules.

A four-digit stamp such as -2507 marks a point release; the families lesson shows why it must be decoded against the repository’s creation date, and tabulates how each family spells every field. A -FP8, -AWQ, -GPTQ or -NVFP4 suffix in the repository name means a pre-quantised safetensors copy published as a separate repository.

A parser makes the convention explicit, and its blank columns show what the name leaves out. It uses only the standard library:

RunnableAll tracks

name-fields.py

"""Split repository identifiers into the fields a name can carry."""
import re

IDS = ["unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M", "Qwen/Qwen3-0.6B-Base",
       "Qwen/Qwen3-8B", "Qwen/Qwen3-30B-A3B-Thinking-2507", "openai/gpt-oss-20b",
       "Qwen/Qwen3-8B-FP8", "google/gemma-4-E4B-it", "Qwen/Qwen3-VL-8B-Instruct",
       "Qwen/Qwen3-Embedding-0.6B", "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B",
       "google/gemma-3-27b-it-qat-q4_0-gguf", "mlx-community/Qwen3-0.6B-bf16"]

VARIANT = {"base": "base", "pt": "base", "instruct": "instruct", "it": "instruct",
           "chat": "instruct", "thinking": "thinking", "coder": "coder", "vl": "vision",
           "embedding": "embedding", "reranker": "reranker", "distill": "distilled",
           "qat": "quant-aware"}
FORMAT = {"gguf", "mlx", "fp8", "awq", "gptq", "nvfp4", "mxfp4", "bf16", "q4_0"}

print(f"{'repository':41} {'total':>5} {'act/eff':>7}  {'variant':20} {'format':10} stamp quant")
for ident in IDS:
    repo, _, quant = ident.partition(":")
    total = active = stamp = "-"
    variant, fmt = [], []
    for part in repo.split("/")[1].lower().split("-"):
        if re.fullmatch(r"\d+(\.\d+)?b", part) and total == "-":
            total = part.upper()
        elif re.fullmatch(r"[ae]\d+(\.\d+)?b", part):     # A = active, E = effective
            active = part.upper()
        elif re.fullmatch(r"\d{4}", part):
            stamp = part
        elif part in VARIANT:
            variant.append(VARIANT[part])
        elif part in FORMAT:
            fmt.append(part)
    print(f"{repo:41} {total:>5} {active:>7}  {'+'.join(variant) or '(none)':20}"
          f" {'+'.join(fmt) or '(none)':10} {stamp:5} {quant or '-'}")

RunnableAll tracks

run name-fields.py

python3 name-fields.py

Output — what you should see

name-fields.py, Python 3.12

repository                                total act/eff  variant              format     stamp quant
unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF   30B     A3B  coder+instruct       gguf       -     Q4_K_M
Qwen/Qwen3-0.6B-Base                       0.6B       -  base                 (none)     -     -
Qwen/Qwen3-8B                                8B       -  (none)               (none)     -     -
Qwen/Qwen3-30B-A3B-Thinking-2507            30B     A3B  thinking             (none)     2507  -
openai/gpt-oss-20b                          20B       -  (none)               (none)     -     -
Qwen/Qwen3-8B-FP8                            8B       -  (none)               fp8        -     -
google/gemma-4-E4B-it                         -     E4B  instruct             (none)     -     -
Qwen/Qwen3-VL-8B-Instruct                    8B       -  vision+instruct      (none)     -     -
Qwen/Qwen3-Embedding-0.6B                  0.6B       -  embedding            (none)     -     -
deepseek-ai/DeepSeek-R1-Distill-Qwen-7B      7B       -  distilled            (none)     -     -
google/gemma-3-27b-it-qat-q4_0-gguf         27B       -  instruct+quant-aware q4_0+gguf  -     -
mlx-community/Qwen3-0.6B-bf16              0.6B       -  (none)               bf16       -     -

Every row is parsed correctly and most of them mislead. Qwen3-8B and gpt-oss-20b (Apache-2.0) have no variant, yet both are post-trained for chat, and the -FP8 and -bf16 copies inherit the unsuffixed post-trained name; mlx-community puts the format in the namespace, where the parser does not look. gpt-oss-20b shows no active count and no format, yet its card gives 21B parameters with 3.6B active, and its weights are MXFP4. gemma-4-E4B-it (Apache-2.0) has no total at all, because E4B counts effective parameters; the model reference records the card’s 8B with embeddings. And DeepSeek-R1-Distill-Qwen-7B (mit in its metadata; the distillation table below reads its card) puts two model names in one field: R1 is the teacher, and Qwen-7B is the student, whose architecture the file actually contains. What each field sets is worth fixing in mind before the variants, because the costs go to different places:

Field	Sets	Worked from this identifier	Taught in
Total parameters	Memory for the weights	30.5B at Q4_K_M: an 18.56 GB file	the memory lesson
Active parameters	Bytes read per decoded token, so decode speed	3.3B of 30.5B, about a ninth, read per token	the architecture lesson
Variant	Which job the post-training fitted it for	Instruct, non-thinking, tool calling	this lesson
Format and tier	Which engines load it, and the file size	GGUF: llama.cpp, Ollama, LM Studio	Part 6
Namespace	Whose files, card and conversion you are trusting	a republication of Qwen’s weights	the next lesson

The evidence behind the name

Four places in a repository confirm or contradict the name, and one script reads all of them without downloading any weights:

Evidence	Where it lives	What it settles
Task and relationship	Hub metadata: pipeline_tag, and tags of the form base_model:<relation>:<repo>	Generation, embedding, ranking or image input; whether this is a fine-tune or a quantisation of another repository
Architecture and extras	config.json: architectures, vision_config, quantization_config	The engine code path, a vision tower, pre-quantised weights
Turn ending and defaults	generation_config.json: eos_token_id, sampling settings	Whether the weights were trained to end a turn
Template and vocabulary	tokenizer_config.json, chat_template.jinja: template variables, added tokens	Which thinking design, and which special tokens exist

The Hub documentation says it “will infer the type of relationship from the current model to the base model ("adapter", "merge", "quantized", "finetune")” unless the card sets it, which is why the script reads the inferred tags rather than the card field. It needs huggingface_hub, which Part 2’s lab installed into ~/llm-course/.venv; on Track S, run it inside Part 2’s container and skip the activate line. The downloads are a few kilobytes of JSON per repository.

RunnableAll tracks

read-the-repo.py

"""Read the variant evidence a repository carries in its metadata and small files."""
import json
from huggingface_hub import HfApi, hf_hub_download

REPOS = ["Qwen/Qwen3-0.6B-Base", "Qwen/Qwen3-0.6B", "Qwen/Qwen3-30B-A3B-Thinking-2507",
         "openai/gpt-oss-20b", "Qwen/Qwen3-Coder-30B-A3B-Instruct", "Qwen/Qwen3-VL-8B-Instruct",
         "Qwen/Qwen3-Embedding-0.6B", "Qwen/Qwen3-Reranker-0.6B",
         "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B", "Qwen/Qwen3-8B-FP8",
         "google/gemma-3-27b-it-qat-q4_0-gguf", "mlx-community/Qwen3-0.6B-bf16"]

def read(repo, files, name):                  # only fetch files the repository has
    return open(hf_hub_download(repo, name)).read() if name in files else ""

for repo in REPOS:
    info = HfApi().model_info(repo)
    files = {s.rfilename for s in info.siblings}
    cfg = json.loads(read(repo, files, "config.json") or "{}")
    gen = json.loads(read(repo, files, "generation_config.json") or "{}")
    tok = json.loads(read(repo, files, "tokenizer_config.json") or "{}")
    vocab = {t["content"] for t in tok.get("added_tokens_decoder", {}).values()}
    template = (str(tok.get("chat_template", "")) + read(repo, files, "chat_template.jinja")
                + read(repo, files, "chat_template.json"))
    thinking = ("effort" if "reasoning_effort" in template else
                "switch" if "enable_thinking" in template else
                "always" if "<think>" in template else "none")
    relation = [t[len("base_model:"):] for t in info.tags
                if t.startswith("base_model:") and t.count(":") == 2]
    extra = sorted(f for f in files if f.startswith(("modules.json", "1_", "mmproj")))
    licence = info.card_data.get("license") if info.card_data else None
    print(f"{repo}  [{licence}, gated={info.gated}]")
    print(f"  task={info.pipeline_tag} arch={(cfg.get('architectures') or ['-'])[0]}"
          f" relation={relation or '-'}")
    print(f"  eos={gen.get('eos_token_id', '-')} thinking={thinking}"
          f" fim_tokens={'<|fim_prefix|>' in vocab} vision={'vision_config' in cfg}"
          f" quant={cfg.get('quantization_config', {}).get('quant_method', '-')}"
          + (f" files={extra}" if extra else ""))

RunnableAll tracks

run read-the-repo.py

source ~/llm-course/.venv/bin/activate
python read-the-repo.py

Output — what you should see

read-the-repo.py, huggingface_hub 1.30.0, run on 2026-09-12 without a token

Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Qwen/Qwen3-0.6B-Base  [apache-2.0, gated=False]
  task=text-generation arch=Qwen3ForCausalLM relation=-
  eos=151643 thinking=switch fim_tokens=True vision=False quant=-
Qwen/Qwen3-0.6B  [apache-2.0, gated=False]
  task=text-generation arch=Qwen3ForCausalLM relation=['finetune:Qwen/Qwen3-0.6B-Base']
  eos=[151645, 151643] thinking=switch fim_tokens=True vision=False quant=-
Qwen/Qwen3-30B-A3B-Thinking-2507  [apache-2.0, gated=False]
  task=text-generation arch=Qwen3MoeForCausalLM relation=-
  eos=[151645, 151643] thinking=always fim_tokens=True vision=False quant=-
openai/gpt-oss-20b  [apache-2.0, gated=False]
  task=text-generation arch=GptOssForCausalLM relation=-
  eos=[200002, 199999, 200012] thinking=effort fim_tokens=False vision=False quant=mxfp4
Qwen/Qwen3-Coder-30B-A3B-Instruct  [apache-2.0, gated=False]
  task=text-generation arch=Qwen3MoeForCausalLM relation=-
  eos=[151645, 151643] thinking=none fim_tokens=True vision=False quant=-
Qwen/Qwen3-VL-8B-Instruct  [apache-2.0, gated=False]
  task=image-text-to-text arch=Qwen3VLForConditionalGeneration relation=-
  eos=[151645, 151643] thinking=none fim_tokens=True vision=True quant=-
Qwen/Qwen3-Embedding-0.6B  [apache-2.0, gated=False]
  task=feature-extraction arch=Qwen3ForCausalLM relation=['finetune:Qwen/Qwen3-0.6B-Base']
  eos=151643 thinking=switch fim_tokens=True vision=False quant=- files=['1_Pooling/config.json', 'modules.json']
Qwen/Qwen3-Reranker-0.6B  [apache-2.0, gated=False]
  task=text-ranking arch=Qwen3ForCausalLM relation=['finetune:Qwen/Qwen3-0.6B-Base']
  eos=[151645, 151643] thinking=switch fim_tokens=True vision=False quant=- files=['1_LogitScore/config.json', 'modules.json']
deepseek-ai/DeepSeek-R1-Distill-Qwen-7B  [mit, gated=False]
  task=text-generation arch=Qwen2ForCausalLM relation=-
  eos=151643 thinking=always fim_tokens=False vision=False quant=-
Qwen/Qwen3-8B-FP8  [apache-2.0, gated=False]
  task=text-generation arch=Qwen3ForCausalLM relation=['quantized:Qwen/Qwen3-8B']
  eos=[151645, 151643] thinking=switch fim_tokens=True vision=False quant=fp8
google/gemma-3-27b-it-qat-q4_0-gguf  [gemma, gated=manual]
  task=image-text-to-text arch=- relation=['quantized:google/gemma-3-27b-it']
  eos=- thinking=none fim_tokens=False vision=False quant=- files=['mmproj-model-f16-27B.gguf']
mlx-community/Qwen3-0.6B-bf16  [apache-2.0, gated=False]
  task=text-generation arch=Qwen3ForCausalLM relation=['finetune:Qwen/Qwen3-0.6B']
  eos=- thinking=switch fim_tokens=True vision=False quant=-

The gated Gemma repository answers without a token because metadata and file names are public; its files are not, and the script fetches none. The script has two blind spots. fim_tokens reads only added_tokens_decoder in tokenizer_config.json, so it reports False for DeepSeek-R1-Distill-Qwen-7B, whose tokenizer.json does carry <|fim_prefix|> at id 151659. For a GGUF repository the arch, eos, thinking, fim_tokens and vision columns print -, none or False whatever the model is, because that evidence is inside the .gguf file; HfApi().model_info(repo, expand=["gguf"]).gguf returns its architecture, context_length, eos_token and chat_template without downloading it (huggingface_hub 1.30.0). The sections below read these lines one variant at a time. Keep the output next to you.

Base, instruct and chat

A base checkpoint is what pretraining produced: it continues documents. A post-trained checkpoint, published as -Instruct, -it, -Chat or -chat, has been through supervised fine-tuning and preference tuning so that a reply in the chat template’s format, ending in an end-of-turn token, is the likely continuation of a question. Part 3 explains those stages and renders the template, and the Part 2 lab showed the base checkpoint answering a quiz question with more quiz questions. Chat and instruct name the same role: -chat and -Chat were the suffixes Llama 2 and Qwen1.5 used, and later generations of both families moved to -Instruct.

The first two blocks of the evidence run are the 0.6B pair from Part 2’s reduced path; the 1.7B pair its primary path downloaded differs in exactly the same fields. The difference is not where you would look first. Both repositories carry the Qwen3 chat template with its enable_thinking switch (the base copy is an earlier revision), so the template is not evidence of post-training. What differs is generation_config.json and the eos_token in tokenizer_config.json:

Evidence	Qwen3-0.6B-Base	Qwen3-0.6B	What it means
eos_token_id	151643, <|endoftext|> only	[151645, 151643], <|im_end|> first	The publisher declares <|im_end|> a stop token only for the post-trained weights
tokenizer_config.json eos_token	<|endoftext|>	<|im_end|>	The same declaration, in the tokeniser’s files
do_sample	false	true	Base defaults to greedy continuation
temperature, top_p, top_k	absent	0.6, 0.95, 20	The card’s thinking-mode sampling settings
max_new_tokens	2048	absent	A cap for open-ended continuation
Hub relation tag	none	finetune:Qwen/Qwen3-0.6B-Base	Lineage, inferred by the Hub
Chat template	an earlier revision of the same template, 4,116 characters	4,168 characters	Not evidence either way

The suffix convention also runs in opposite directions, so an unsuffixed name tells you nothing until you know the family. Licences below are from each repository’s card metadata on 2026-09-12:

Generation	Base checkpoint	Post-trained checkpoint	Licence, gating	The unsuffixed name is
Llama 2 (2023)	Llama-2-7b-hf	Llama-2-7b-chat-hf	Llama 2 licence, gated	base
Qwen1.5 (2024)	Qwen1.5-7B	Qwen1.5-7B-Chat	other (license_name tongyi-qianwen)	base
Qwen2 (2024)	Qwen2-7B	Qwen2-7B-Instruct	Apache-2.0	base
Llama 3.1 (2024)	Llama-3.1-8B	Llama-3.1-8B-Instruct	Llama 3.1 Community licence, gated	base
Gemma 3 (2025)	gemma-3-27b-pt	gemma-3-27b-it	Gemma terms, gated	not used
Qwen3 (2025)	Qwen3-8B-Base	Qwen3-8B	Apache-2.0	post-trained
SmolLM3 (2025)	SmolLM3-3B-Base	SmolLM3-3B	Apache-2.0	post-trained
Gemma 4 (2026)	gemma-4-E4B	gemma-4-E4B-it	Apache-2.0	base

The decision is short. Download the post-trained checkpoint for anything interactive (chat, extraction, tool calls, an API), and also as the starting point for a LoRA fine-tune to your own format, which is what Part 13 does. Take -Base only when you will replace the chat format or run the post-training stages yourself.

✕Common mistake

A base checkpoint served as a chat model is not broken; it continues your message. Under Transformers generate(), which takes its stop ids from generation_config.json, Qwen3-0.6B-Base stops only on 151643, so an <|im_end|> it writes does not end the turn and it runs on into new text until <|endoftext|> or max_new_tokens, 2048 in that file. llama-server hides the difference: at llama.cpp v0.4.0 · verified 2026-09-08, llama-vocab.cpp adds every token whose text is <|im_end|> or <|endoftext|> to its end-of-generation set, whatever the repository declares, so the stop reason does not identify a base model. Neither does the GGUF’s own EOS: the file headers of unsloth/Qwen3-0.6B-GGUF and of mradermacher/Qwen3-0.6B-Base-GGUF, a community base conversion the course does not download, both record id 151645, <|im_end|> (read 2026-09-13), and the load log prints a line of the form print_info: EOS token = 151645 '<|im_end|>' for either. For a GGUF, follow the repository’s quantized: relation tag to its source and read eos_token_id there, before you touch a sampling setting.

Thinking modes and reasoning budgets

A thinking mode is post-training that makes the model write working before its answer, inside marked tokens a client can separate from the answer. A reasoning budget caps how many tokens that working may take. The working is ordinary generated text, Part 3 prices it in seconds, and Part 10 drives the switches through an engine. What this lesson adds is which design a repository has and how to budget for it, because the name gives it away for only one of four designs:

Design	Example, licence	Signature in the evidence run	How it is set	Output length the card recommends
A switch in one checkpoint	Qwen3-8B, SmolLM3-3B; Apache-2.0	thinking=switch: the template reads enable_thinking	enable_thinking=False in the template, or /think and /no_think in a message	Qwen3-8B: 32,768 tokens; 38,912 for competition maths and programming
An effort dial	gpt-oss-20b; Apache-2.0	thinking=effort: the template reads reasoning_effort, default medium	Reasoning: low, medium or high in the system prompt	not stated
A thinking-only repository	Qwen3-30B-A3B-Thinking-2507; Apache-2.0	thinking=always: the generation prompt already opens <think>	choose the repository	32,768; 81,920 for highly challenging tasks
No thinking	Qwen3-30B-A3B-Instruct-2507, Qwen3-Coder-30B-A3B-Instruct; Apache-2.0	thinking=none	choose the repository	16,384 and 65,536

The thinking-only repository’s pair, -Instruct-2507, has the same Qwen3MoeForCausalLM architecture and is, in its card’s words, “the updated version of the Qwen3-30B-A3B non-thinking mode”. Render one question through three templates to see where each design lives. This needs transformers, from the same environment, and downloads only tokeniser files, a few tens of megabytes:

RunnableAll tracks

thinking-prompts.py

"""Where each thinking design lives: render one question through three templates."""
from transformers import AutoTokenizer

messages = [{"role": "user", "content": "What is 17 * 23?"}]

def render(repo, **template_kwargs):
    tok = AutoTokenizer.from_pretrained(repo)
    text = tok.apply_chat_template(messages, tokenize=False,
                                   add_generation_prompt=True, **template_kwargs)
    return text, len(tok(text, add_special_tokens=False)["input_ids"])

for effort in ("low", "medium", "high"):
    text, n = render("openai/gpt-oss-20b", reasoning_effort=effort)
    line = next(l for l in text.splitlines() if l.startswith("Reasoning:"))
    print(f"gpt-oss-20b, reasoning_effort={effort:6}: {n} prompt tokens, {line!r}")

for repo in ("Qwen/Qwen3-30B-A3B-Thinking-2507", "Qwen/Qwen3-30B-A3B-Instruct-2507"):
    text, n = render(repo)
    print(f"{repo.split('/')[1]}: {n} prompt tokens, ends {text[-30:]!r}")

RunnableAll tracks

run thinking-prompts.py

python thinking-prompts.py

Output — what you should see

thinking-prompts.py, transformers 5.16.1, run on 2026-09-12

Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
gpt-oss-20b, reasoning_effort=low   : 75 prompt tokens, 'Reasoning: low'
gpt-oss-20b, reasoning_effort=medium: 75 prompt tokens, 'Reasoning: medium'
gpt-oss-20b, reasoning_effort=high  : 75 prompt tokens, 'Reasoning: high'
Qwen3-30B-A3B-Thinking-2507: 20 prompt tokens, ends '<|im_start|>assistant\n<think>\n'
Qwen3-30B-A3B-Instruct-2507: 18 prompt tokens, ends 'm_end|>\n<|im_start|>assistant\n'

The gpt-oss template writes today’s date into its system message, so the count can move by a token on another day. The effort dial costs nothing in the prompt: the three prompts are the same length and differ in one word, so any difference in cost is in what the model then generates, which the card describes as “Fast responses for general dialogue” at low and “Deep and detailed analysis” at high. The thinking-only template spends its two extra tokens opening the block for the model.

A budget is enforced from outside the weights. The Qwen3 technical report describes the mechanism: “when the length of the model’s thinking reaches a user-defined threshold, we manually halt the thinking process and insert the stop-thinking instruction”, and notes that this “is not explicitly trained but emerges naturally” from training both modes into one model. llama-server exposes the same shape: its README documents --reasoning-budget N as “-1 for unrestricted, 0 for immediate end, N>0 for token budget”, with --reasoning-budget-message injected “before the end-of-thinking tag when reasoning budget is exhausted” (read at v0.4.0 on 2026-09-12; Part 24 tests it in a loop).

Budget memory as well as time. Every output token stays in the key-value cache until the turn ends, at the bytes per token that the families lesson reads off config.json:

Pseudocode — not a real command

output_cache_bytes = output_tokens × bytes_per_token
bytes_per_token    = 2 × layers × kv_heads × head_dim × bytes_per_element    (2: one key and one value)

Qwen3-8B:       2 × 36 × 8 × 128 × 2 (FP16) = 147,456
Qwen3-30B-A3B:  2 × 48 × 4 × 128 × 2 (FP16) =  98,304    (Thinking-2507, Instruct-2507, Coder)

Arithmetic from those inputs and the cards’ recommended lengths, for the output alone, on top of the prompt’s own cache:

Model and setting	Output tokens	Bytes per token	Cache for the output
Thinking-2507, the thinking budget its long-context evaluation used	8,192	98,304	0.81 GB
Instruct-2507, recommended output length	16,384	98,304	1.61 GB
Thinking-2507, recommended output length	32,768	98,304	3.22 GB
Qwen3-8B, thinking on, recommended output length	32,768	147,456	4.83 GB
Qwen3-8B, competition problems	38,912	147,456	5.74 GB
Thinking-2507, highly challenging tasks	81,920	98,304	8.05 GB

The rule that follows: set the context length to the prompt plus the output length the card recommends for the mode you run, price that with the formula before you enable thinking in a service, and if it does not fit, lower the budget rather than let generation hit the context limit inside the working, where the answer is never written.

✕Common mistake

A parser that looks for <think> in the output of Qwen3-30B-A3B-Thinking-2507 or DeepSeek-R1-Distill-Qwen-7B finds nothing, treats the whole reply as the answer, and shows the user the working. Their templates put the opening tag in the prompt, and the Thinking-2507 card warns “it is normal for the model’s output to contain only </think>”. Split on the last </think>, as the card’s own example does with token 151668.

Coder models and fill in the middle

A coder model is post-trained, and usually mid-trained, on code and the work around it: patches, tests, tool calls, repository-scale context. Fill in the middle (FIM) is a separate trained ability: given the text before and after a gap, produce the gap, which is what editor completion needs. The prompt lays the pieces out as prefix, suffix, then middle, with three control tokens. The Qwen3-Coder README gives the layout, and Part 10 wires it to an editor:

Pseudocode — not a real command

prompt = '<|fim_prefix|>' + prefix_code + '<|fim_suffix|>' + suffix_code + '<|fim_middle|>'

The README states that “FIM is supported in every version of Qwen3-Coder”; its example sends that string as a user message through the chat template, with the system message “You are a code completion assistant.”, and stops generation on the FIM and end-of-turn ids. The Qwen3-Coder-30B-A3B-Instruct card itself does not mention FIM, which is why the README is the source here.

Now read the fim_tokens column of the evidence run. The FIM tokens are present in every Qwen3 tokeniser, base, embedding, reranker and vision models included, at ids 151659 to 151664 as Part 2’s tokeniser lesson listed them. One tokeniser serves the whole family, so the tokens prove nothing about training. Engines cannot tell the difference either: llama.cpp v0.4.0 · verified 2026-09-08 finds FIM tokens by their text when it loads a vocabulary (<|fim_prefix|> is matched with the comment // Qwen in llama-vocab.cpp) and logs a line of the form print_info: FIM PRE token = 151659 '<|fim_prefix|>' for any Qwen3 GGUF, and the server’s /infill handler, in server-context.cpp, checks only that the prefix, suffix and middle tokens exist. A vocabulary without them is refused with an error beginning “Infill is not supported by this model”; every Qwen3 model is accepted and completes the gap as well as its training allows. Require a document that states FIM for the model you will run.

The other half of a coder’s value is context, and context is memory. Qwen3-Coder-30B-A3B-Instruct has a native context of 262,144 tokens, and its card advises “reducing the context length to a shorter value, such as 32,768” on out-of-memory errors. Arithmetic from the Q4_K_M file size in the Hub listing and the 98,304 bytes per token at FP16 from the thinking section’s formula:

Context length	KV cache at FP16	Weights, Q4_K_M	Weights and cache
32,768	3.22 GB	18.56 GB	21.78 GB
65,536	6.44 GB	18.56 GB	25.00 GB
131,072	12.88 GB	18.56 GB	31.44 GB
262,144	25.77 GB	18.56 GB	44.33 GB

The native window costs more memory than the weights. The last column leaves out the operating system and compute buffers, which the memory lesson’s headroom rules add. The decision by job:

Job	Choose	Require as evidence
An agent editing a repository	A coder instruct model, non-thinking or low effort	A documented tool-call format (this repository ships qwen3coder_tool_parser.py) and a context that fits the table above
Ghost-text completion in an editor	A small model that documents FIM	The FIM statement in a card or README, not the tokens
Explaining or reviewing code in chat	Any instruct model; a coder if it fits	Nothing beyond the usual card reading

Vision-language models

A vision-language model accepts images, and sometimes video, alongside text: a vision encoder turns pixels into vectors, a projector maps them into the language model’s embedding space, and from there they are context tokens. Part 10 owns the stack and the engines. The name marks it inconsistently: VL in Qwen3-VL-8B-Instruct (Apache-2.0), nothing at all in gemma-3-27b-it (Gemma terms, gated). The evidence-run line for Gemma’s QAT GGUF repository, gemma-3-27b-it-qat-q4_0-gguf, says task=image-text-to-text and lists an mmproj file, while vision=False only because a GGUF repository has no config.json. In a safetensors repository the signature is a vision_config beside the text_config (the architecture is Qwen3VLForConditionalGeneration); in a GGUF repository it is a separate mmproj-*.gguf projector file, which llama-server --hf-repo downloads “automatically if available”, per the README. Load the language file without it and the text works while an image does not: at v0.4.0, server-common.cpp rejects a chat request carrying an image with “image input is not supported - hint: if this is unexpected, you may need to provide the mmproj”.

Two numbers the name leaves out are the vision tower’s share of the parameters and the tokens each image costs. The safetensors headers give the first without downloading weights, and the preprocessor configuration gives the second:

RunnableAll tracks

vision-cost.py

"""What the 8B in Qwen3-VL-8B leaves out: the vision tower's parameters and each image's tokens."""
import json, math
from collections import Counter
from huggingface_hub import HfApi, hf_hub_download

REPO = "Qwen/Qwen3-VL-8B-Instruct"
params = Counter()
for file_meta in HfApi().get_safetensors_metadata(REPO).files_metadata.values():
    for name, tensor in file_meta.tensors.items():
        params[".".join(name.split(".")[:2])] += tensor.parameter_count
for part, count in sorted(params.items()):
    print(f"{part:22} {count:>13,}  {count / sum(params.values()):6.1%}")

pre = json.load(open(hf_hub_download(REPO, "preprocessor_config.json")))
factor = pre["patch_size"] * pre["merge_size"]      # pixels per side of one image token
lo, hi = pre["size"]["shortest_edge"], pre["size"]["longest_edge"]   # pixel-count limits

def image_tokens(h, w):                              # transformers' smart_resize, then 32x32 blocks
    hb, wb = round(h / factor) * factor, round(w / factor) * factor
    if hb * wb > hi:
        beta = math.sqrt(h * w / hi)
        hb, wb = math.floor(h / beta / factor) * factor, math.floor(w / beta / factor) * factor
    elif hb * wb < lo:
        beta = math.sqrt(lo / (h * w))
        hb, wb = math.ceil(h * beta / factor) * factor, math.ceil(w * beta / factor) * factor
    return (hb // factor) * (wb // factor)

KV_BYTES_PER_TOKEN = 2 * 36 * 8 * 128 * 2           # text_config: layers, KV heads, head_dim; FP16
for h, w in [(480, 640), (1080, 1920), (2200, 1700), (3024, 4032), (6000, 8000)]:
    n = image_tokens(h, w)
    print(f"{w}x{h}: {n:>6,} tokens, {n * KV_BYTES_PER_TOKEN / 1e9:5.2f} GB of KV cache at FP16")

RunnableAll tracks

run vision-cost.py

python vision-cost.py

Output — what you should see

vision-cost.py, huggingface_hub 1.30.0, run on 2026-09-12

Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Parse safetensors files: 100%|██████████| 4/4 [00:00<00:00,  x.xxit/s]
lm_head.weight           622,329,856    7.1%
model.language_model   7,568,405,504   86.3%
model.visual             576,388,336    6.6%
640x480:    300 tokens,  0.04 GB of KV cache at FP16
1920x1080:  2,040 tokens,  0.30 GB of KV cache at FP16
1700x2200:  3,657 tokens,  0.54 GB of KV cache at FP16
4032x3024: 11,844 tokens,  1.75 GB of KV cache at FP16
8000x6000: 16,170 tokens,  2.38 GB of KV cache at FP16

The 8B is the whole checkpoint, 8.77 billion parameters, of which the vision tower is 576 million. Each image token covers a 32 × 32 pixel block (patch_size 16 × merge_size 2), after the image is resized to multiples of 32 and to between 65,536 and 16,777,216 pixels, so a 12-megapixel phone photo costs 11,844 tokens and the largest images stop at about 16,000. The smart_resize logic is copied from transformers 5.16.1 · verified 2026-09-08 and checked against it; other engines may resize differently, so treat the token counts as the Transformers processor’s defaults. The GGUF repository splits the same checkpoint into files, from its Hub listing on 2026-09-12:

File in unsloth/Qwen3-VL-8B-Instruct-GGUF	Bytes	Needed for
Qwen3-VL-8B-Instruct-Q4_K_M.gguf	5,027,785,568	Text, and the language half of every image request
mmproj-F16.gguf	1,159,030,336	Images and video
mmproj-BF16.gguf	1,162,569,280	The same projector at BF16
mmproj-F32.gguf	2,305,574,464	The same projector at 32-bit, for engines or checks that want it

The Q4_K_M language file is within 1,056 bytes of unsloth/Qwen3-8B-GGUF’s Qwen3-8B-Q4_K_M.gguf, so budget a vision model as its text sibling plus the projector plus the image tokens at the resolution you actually send.

Embedding and reranker models

An embedding model turns a text into one fixed-length vector so that texts which should match land close together; it is a bi-encoder, because the query and each document are encoded separately. A reranker reads the query and one candidate document together and returns one relevance score; it is a cross-encoder. Neither writes text. Part 2 showed that Qwen3-Embedding-0.6B’s config.json matches the generative Qwen3-0.6B’s down to the head counts, and measured what the wrong pooling does. The evidence run adds the lineage, both are tagged finetune:Qwen/Qwen3-0.6B-Base, and shows that the job is recorded in the Hub task and in the Sentence Transformers module files:

Kind	Example (Apache-2.0)	Hub task	What the last layer’s output becomes	Recorded in	Forward passes per query
Generative	Qwen3-0.6B	text-generation	151,936 logits at the last position, sampled into a token	generation_config.json	one per generated token
Embedding	Qwen3-Embedding-0.6B	feature-extraction	The last token’s 1,024 numbers, normalised	modules.json: Transformer, Pooling, Normalize; 1_Pooling/config.json: pooling_mode_lasttoken true	one for the query; documents were embedded once, at indexing
Reranker	Qwen3-Reranker-0.6B	text-ranking	Two logits, for yes (id 9693) and no (id 2152), turned into P(yes)	modules.json: Transformer, LogitScore; 1_LogitScore/config.json: true_token_id 9693, false_token_id 2152	one per candidate

The reranker is a chat model with one job. Its card’s usage code wraps each pair in a system message, “Judge whether the Document meets the requirements based on the Query and the Instruct provided. Note that the answer can only be "yes" or "no".”, opens the assistant turn with an empty thinking block, and reads the yes and no logits at the last position. That is why it is accurate and why it is expensive: every candidate is a full prefill. Counted with the reranker’s tokeniser, the fixed framing is about 73 tokens: 39 for the system message, 25 for the default instruction and the three labels, 9 for the suffix. Token boundaries can move that by one or two:

Pseudocode — not a real command

reranker prefill per query  ≈  K × (73 + query_tokens + chunk_tokens)      K = candidates reranked
embedding prefill, once     ≈  N × (chunk_tokens + 1)                      N = chunks; <|endoftext|> appended

Candidates reranked	Query 7 tokens, chunks 400 tokens	Reranker prefill per query
20	20 × 480	9,600 tokens
50	50 × 480	24,000 tokens
100	100 × 480	48,000 tokens

Indexing 10,000 such chunks is 4,010,000 tokens of prefill once; each query then costs one short embedding pass plus the reranker line you chose. Engines keep the three kinds apart: the llama-server README at v0.4.0 describes --embedding as “use only with dedicated embedding models”, and its /reranking section reads “Requires a reranker model” and “the --embedding --pooling rank options”. Part 10 builds the pipeline and chooses K. One rule comes with the name: an index belongs to the embedding model that built it, so changing that model means re-embedding every chunk.

Distilled, quantisation-aware and merely quantised

Four operations produce repositories that look alike and differ in what happened to the weights.

• Distilled: a smaller student was trained on a larger teacher’s outputs, so the weights are new. The architecture and base vocabulary are the student’s; the distiller may replace special tokens and the chat template, as the DeepSeek-R1-Distill card says (“We slightly change their configs and tokenizers. Please use our setting to run these models.”).
• Quantisation-aware: the model was trained or fine-tuned with rounding in the forward pass, so the weights were adjusted to survive a specific low-precision format.
• Quantised: existing weights were rounded after training, by the publisher or by a third party, sometimes guided by an importance matrix. Nothing was retrained.
• Converted: the same numbers were written into another format, such as MLX at bf16.

Part 3 teaches the mechanisms, and Part 16 reads a QAT release in detail. Here, the name against the evidence, with sizes from the Hub listings on 2026-09-12:

Repository	What was done, per its card or report	Evidence run	Licence: metadata, then what the card adds	Weights on disk
deepseek-ai/DeepSeek-R1-Distill-Qwen-7B	Base model Qwen2.5-Math-7B, “finetuned with 800k samples curated with DeepSeek-R1”	Qwen2ForCausalLM, no relation tag, thinking=always	mit; the card adds that the Qwen-based students derive from Qwen2.5, “originally licensed under Apache 2.0 License”	15.23 GB safetensors
Qwen/Qwen3-8B	Strong-to-weak distillation from Qwen3-32B or Qwen3-235B-A22B, per the technical report, for 0.6B to 14B and 30B-A3B	finetune:Qwen/Qwen3-8B-Base	Apache-2.0	16.38 GB safetensors
openai/gpt-oss-20b	“post-trained with MXFP4 quantization of the MoE weights”	quant=mxfp4	Apache-2.0	13.76 GB safetensors
google/gemma-3-27b-it-qat-q4_0-gguf	Quantisation-aware training for Q4_0	quantized:google/gemma-3-27b-it, mmproj file	gemma, gated	17.23 GB plus 0.86 GB projector
Qwen/Qwen3-8B-FP8	“fine-grained fp8 quantization with block size of 128”, by the publisher	quantized:Qwen/Qwen3-8B, quant=fp8	Apache-2.0	9.44 GB safetensors
unsloth/Qwen3-8B-GGUF	A third party’s quantisation; the repository ships imatrix_unsloth.dat	quantized:Qwen/Qwen3-8B in the Hub tags	Apache-2.0	Qwen3-8B-Q4_K_M.gguf, 5.03 GB
mlx-community/Qwen3-0.6B-bf16	A format conversion at the source’s precision	finetune:Qwen/Qwen3-0.6B	Apache-2.0	model.safetensors, 1.19 GB

Three readings from that table. First, Qwen3-8B is a distilled model and its name does not say so, while DeepSeek-R1-Distill-Qwen-7B is one and its name does; the word in the name is a publisher’s choice. For a distilled model the architecture field tells you what an engine and a fine-tuning script will see: Qwen2, not DeepSeek-R1. The special tokens are another matter. Both config.json files give vocab_size 152,064, but in tokenizer.json id 151644 is <|im_start|> in Qwen2.5-Math-7B (Apache-2.0) and <｜User｜> in the distill, whose template opens the reply with <｜Assistant｜><think>; a Qwen chat template applied to it formats the prompt wrongly. Second, “quantised” covers work of very different care. A QAT checkpoint was trained for its format, and a community tier was rounded with a stated method: the bartowski Llama 3.1 8B repository names llama.cpp release b3472 and states “All quants made using imatrix option”, and the unsloth repositories describe their own “Dynamic 2.0” scheme. Neither is the publisher’s own file, so compare a score only against the same file. Third, the Hub’s relation tag is inferred, and the MLX conversion shows it can be wrong: a bf16 re-export is not a fine-tune.

✕Common mistake

Card metadata is a summary, not the licence. deepseek-ai/DeepSeek-R1-Distill-Llama-8B also carries license: mit in its metadata, while its card states it “is derived from Llama3.1-8B-Base and is originally licensed under llama3.1 license”. The Hub documentation says users “will be able to filter models by license” from that metadata field. Read the card’s licence section and the base model’s licence text before weights go into a product.

Formats, which are not variants

The file format is orthogonal to everything above: the same Instruct weights exist as safetensors, GGUF and MLX, and each engine loads only some of them. safetensors, in its documentation’s words, is a format for storing tensors safely, as opposed to pickle, while remaining fast through zero-copy loading. GGUF, per the Hub’s GGUF page, carries the tensors and a standard set of metadata in one file, which is why a GGUF needs no separate tokeniser or template file. MLX repositories, mostly under mlx-community, are safetensors laid out for mlx-lm. Recognise them from the file list:

Format	Loaded by	Signature in the file list	Precision is recorded in	Example, Hub listing on 2026-09-12
safetensors	Transformers, vLLM, SGLang (Part 9, Parts 11 to 17)	model.safetensors, or model-00001-of-00005.safetensors shards with model.safetensors.index.json, plus config.json and tokeniser files	config.json: torch_dtype; quantization_config.quant_method when pre-quantised	Qwen/Qwen3-8B: 16.38 GB in 5 shards; Qwen/Qwen3-8B-FP8: 9.44 GB in 2
GGUF	llama.cpp, Ollama, LM Studio (Part 6, Part 7)	one <Model>-<TIER>.gguf per tier; mmproj-*.gguf for vision	the tier in the file name, and the tensor types inside the file	Qwen3-8B-Q4_K_M.gguf: 5.03 GB; Qwen3-8B-Q8_0.gguf: 8.71 GB
MLX	mlx-lm (Part 8)	model.safetensors and config.json under mlx-community	config.json: quantization with group_size and bits; -4bit or -bf16 in the name	mlx-community/Qwen3-8B-4bit: group_size 64, bits 4

A model is not available for your engine until its format exists. A new architecture often lands as safetensors only, and the GGUF conversion waits on llama.cpp support for it, which is one of the checks in the families lesson’s refresh procedure.

What the name does not tell you

Each of these looks as though the name should settle it, and each has a file or a measurement that does:

Question	Where the answer lives	Evidence from this lesson
The licence	Card metadata, then the card’s licence section and the licence text	Qwen1.5 other (tongyi-qianwen) but Qwen2 Apache-2.0; Gemma 3 gated, Gemma 4 Apache-2.0; the R1-Distill-Llama metadata against its card
The context window	The card’s native and scaled figures; max_position_embeddings only for the engine default	Qwen3-8B: 32,768 native, 131,072 with YaRN, config 40,960
Whether tool calling is reliable	Nothing static: a measurement at your quantisation	Part 24 measures it
Who quantised it, and how	Namespace, relation tag, the card’s method, files such as an importance matrix	Qwen3-8B-Q4_K_M.gguf is 5,027,783,488 bytes in Qwen/Qwen3-8B-GGUF (5 tiers) and 5,027,784,512 in unsloth/Qwen3-8B-GGUF (25 tiers); the course uses the second because one namespace ships every tier it needs

The name narrows the search to a handful of repositories. The files decide, and the next lesson reads the card’s claims about quality the same way.

Choosing the variant for the job

From the job to the variant

1. 1Turning passages into vectors for searchAn embedding model (task feature-extraction), plus a reranker (text-ranking) for the top candidates. Budget K candidates of prefill per query.
2. 2Reading images, screenshots or documentsA vision-language model: vision_config or an mmproj file, with the projector and the image tokens in the memory budget.
3. 3Editing code in an agent loopA coder instruct model with a documented tool-call format, at the longest context the KV arithmetic fits.
4. 4Autocomplete inside an editorA model whose card or README states fill-in-the-middle. The tokens in the vocabulary are not evidence.
5. 5Hard problems with checkable answers, latency acceptableA thinking switch turned on, a high effort level or a Thinking repository, with the card output length in the context budget.
6. 6General chat, summarising, extractionA post-trained model with thinking off or effort low. Most work, and the cheapest setting.
7. 7Fine-tuning to your formatThe post-trained checkpoint (Part 13); a Base checkpoint only when you replace the chat format, confirmed by eos_token_id.

Work down the list; the first match is your answer. Then confirm it with the evidence run: task tag, architecture, eos ids, template variables and extra files.

Resolve a name into an executable identity

Read a model name as a collection of hints that must be resolved into files. A suffix may describe a training stage, quantisation recipe, context extension or contributor convention. It does not necessarily identify the upstream base revision, tokeniser or conversion settings. Two repositories with similar filenames can therefore produce different behaviour.

Write an identity record with the repository, revision, exact filename or shard set, base lineage, tokeniser, chat template and engine version. Use the model’s published configuration to confirm architecture and parameter interpretation. For a conversion, preserve the converter revision and command so another reader can distinguish a publisher checkpoint from your derived artefact.

Try explaining an ambiguous filename to a colleague without using the phrase “the latest one”. If you cannot specify the bytes they should load, the record is incomplete. This matters during rollback: replacing a file behind an unchanged gateway alias can silently change results. Keep the friendly alias for applications, but map it to an immutable identity in the deployment record.

Recap

Each field of an identifier maps to a file or a Hub tag that confirms or contradicts it. The decisive evidence differs by variant: eos_token_id for base against post-trained, template variables for thinking, a stated FIM ability rather than FIM tokens, vision_config or an mmproj file for images, module files for embedding and reranking, and the card and tokenizer.json for a distillation. Budget each choice in memory or tokens before you run it: KV cache per output token for thinking, the projector plus image tokens for vision, and K prefills per query for a reranker.

Check your understanding

Question 1. A Qwen3 checkpoint run with Transformers generate() writes <|im_end|> after its reply and keeps going, into new questions, until max_new_tokens. Which file confirms that you downloaded the base checkpoint?

• tokenizer_config.json, because its chat template contains enable_thinking
• generation_config.json, because eos_token_id is 151643 alone
• config.json, because architectures is Qwen3ForCausalLM
• The model name, because it has no -Instruct suffix

Show the answer and why

Answer: generation_config.json, because eos_token_id is 151643 alone

generate() takes its stop ids from generation_config.json, and only the post-trained file lists 151645, <|im_end|>. tokenizer_config.json does differ, in eos_token, but not for the reason given: both repositories carry a template with enable_thinking. Both share the architecture, and in Qwen3 the unsuffixed name is the post-trained model, the opposite of Llama 3.1 and Gemma 4. Under llama-server this symptom would not appear, because llama.cpp stops on <|im_end|> by its text in any vocabulary that contains it.

Question 2. You raise the output length for Qwen3-30B-A3B-Thinking-2507 from the card's 32,768 tokens to its 81,920 for hard tasks. At 98,304 bytes per token of FP16 cache, how much more memory does the output alone need?

• About 0.49 GB
• About 4.83 GB
• About 8.05 GB
• Nothing: thinking tokens are discarded, so they are not cached

Show the answer and why

Answer: About 4.83 GB

49,152 more tokens × 98,304 bytes = 4.83 GB, taking the output cache from 3.22 GB to 8.05 GB. The cards advise dropping the working from later turns' history, but during the turn every thinking token sits in the KV cache like any other, so the context length has to hold the prompt plus the whole output.

Question 3. This parser handles replies from Qwen3-30B-A3B-Thinking-2507. What goes wrong?

if "<think>" in reply:
    reasoning, answer = reply.split("</think>", 1)
else:
    reasoning, answer = "", reply

• Nothing; it is correct for every thinking model
• split with maxsplit 1 keeps the last </think> in the answer
• The template already opened <think> in the prompt, so the reply contains only </think>; the else branch runs and the working is shown as the answer
• The model never emits </think>, so the split raises an exception

Show the answer and why

Answer: The template already opened <think> in the prompt, so the reply contains only </think>; the else branch runs and the working is shown as the answer

The evidence run labels this design thinking=always: the generation prompt already opens the thinking block, and the card says the output normally contains only the closing tag. Test for the closing tag and split on its last occurrence, as the card's own example does with token 151668. DeepSeek-R1-Distill-Qwen-7B's template does the same.

Question 4. A Qwen3-8B GGUF loads in llama-server and the log prints a FIM PRE token line for id 151659. What does that establish?

• The model was trained for fill-in-the-middle and /infill will work well
• The tokeniser contains <|fim_prefix|>, which llama.cpp found by its text; it says nothing about whether the weights were trained for FIM
• The file is a Qwen3-Coder model with the wrong name
• The GGUF was converted with an importance matrix

Show the answer and why

Answer: The tokeniser contains <|fim_prefix|>, which llama.cpp found by its text; it says nothing about whether the weights were trained for FIM

Every Qwen3 tokeniser carries ids 151659 to 151664, including the base, embedding, reranker and vision repositories, and llama-vocab.cpp matches the token text for any model. The Qwen3-Coder README states FIM support for Qwen3-Coder; require a statement like that for the model you run.

Question 5. Which statements about vision-language repositories are correct? Select all that apply.

• The Qwen3-VL-8B-Instruct Q4_K_M GGUF file alone is enough to send images
• At the Transformers processor defaults, a 4032 × 3024 photo costs about 11,844 tokens of context
• gemma-3-27b-it cannot read images because its name has no VL marker
• A pipeline_tag of image-text-to-text, a vision_config in config.json or an mmproj file each indicates image input

Show the answer and why

Answer: At the Transformers processor defaults, a 4032 × 3024 photo costs about 11,844 tokens of context, A pipeline_tag of image-text-to-text, a vision_config in config.json or an mmproj file each indicates image input

Images need the projector, a separate mmproj file of about 1.16 GB at F16 alongside the 5.03 GB language file. Each image token covers 32 × 32 pixels after resizing, so 3008 × 4032 resized pixels make 94 × 126 = 11,844 tokens, 1.75 GB of FP16 cache on this model. Gemma 3 27B is image-text-to-text with no marker in its name.

Question 6. DeepSeek-R1-Distill-Qwen-7B's config.json names Qwen2ForCausalLM, and its tokenizer.json maps id 151644 to <｜User｜>. What follows?

• The file is mislabelled; a DeepSeek-R1 student should load as a DeepSeek architecture
• The weights are DeepSeek-R1 rounded to 7B parameters
• Engines and fine-tuning scripts treat it as a Qwen2-architecture model with Qwen's base vocabulary but DeepSeek's special tokens and chat template, and its weights are new, trained on teacher-generated samples
• It is a quantisation of Qwen2.5-Math-7B with no training

Show the answer and why

Answer: Engines and fine-tuning scripts treat it as a Qwen2-architecture model with Qwen's base vocabulary but DeepSeek's special tokens and chat template, and its weights are new, trained on teacher-generated samples

Distillation trains the student base, here Qwen2.5-Math-7B, on 800k samples curated with DeepSeek-R1. The architecture, the base vocabulary and engine support come from the student; in Qwen2.5-Math-7B id 151644 is <|im_start|>, so the distiller replaced the special tokens and the template, and a Qwen chat template applied to the distill formats the prompt wrongly. The licence needs the card as well as the metadata: mit in the metadata, and an Apache-2.0 origin in the card.

Check my answersClear