Aider with Local Models

Aider is the shortest path from a local server to an agent that edits your repository, and it is the only tool in this part that ships the benchmark it is judged by. By the end of this lesson you will have it running against your Part 9 gateway, you will know which edit format your model can actually produce and why that is the setting that decides whether the tool works, and you will know how to run its polyglot benchmark yourself rather than reading someone else’s number.

The course pins Aider 0.86.0 · verified 2026-09-08. Its repository ships the Apache-2.0 licence text.

Two variables and a prefix

Aider’s documentation states the compatibility promise directly: “Aider can connect to any LLM which is accessible via an OpenAI compatible API endpoint.” The configuration is two environment variables and a model name.

RunnableAll tracks

point Aider at the Part 9 gateway

export OPENAI_API_BASE=http://127.0.0.1:4000
export OPENAI_API_KEY="${GATEWAY_KEY}"
aider --model openai/local/coder

Three things about that block are worth stating rather than assuming.

The openai/ prefix is not decoration. Aider routes by prefix, and without it the model name is looked up in a table of known hosted models and either fails or, worse, resolves to something you did not mean. The part after the prefix is your gateway alias from Part 9, so openai/local/coder reads as “the OpenAI-compatible route, to the model called local/coder”, exactly as it did in the gateway project.

The key is a virtual key you generated on the gateway, not a provider key. If you are running the native gateway path on Track M without a database, it is the master key, and the gateway project says which steps that costs you.

And the base URL points at the gateway rather than at an engine. Change the model behind local/coder next month and this command does not change.

✦Tip

Set both variables in your shell profile and you never type them again; set them in the gateway directory’s .env and source it when you work there, and you keep one place where the key lives. Do not put the key in a shell command that lands in your history file, which is why the block above reads it from a variable rather than containing it.

Against a plain Ollama install, without a gateway, the documentation names a different variable and a different prefix: OLLAMA_API_BASE and the ollama_chat/ prefix, which its Ollama page recommends in preference to ollama/.

RunnableAll tracks

the no-gateway path, straight at Ollama

export OLLAMA_API_BASE=http://127.0.0.1:11434
aider --model ollama_chat/qwen3-coder:30b

The context window is the setting that will bite you

Aider’s Ollama page is unusually blunt about this: “Ollama defaults to a 2k context window and ‘silently’ discards excess context”. Silently is the operative word. Nothing errors. The model simply stops seeing the beginning of the conversation, and you conclude it is stupid.

There are two fixes and you should understand both. OLLAMA_CONTEXT_LENGTH set before ollama serve raises the server’s default. And a per-model entry in .aider.model.settings.yml fixes the value Aider requests, which the documentation shows as num_ctx under extra_params.

The same trap exists behind a gateway, one layer down. If the engine behind local/coder was started with a small --ctx-size, the alias inherits it, and Aider cannot see that. This is the first thing to check when a local model behaves worse in Aider than it did in a chat window.

RunnableAll tracks

aider-model-settings.yml

# Purpose: per-model settings for Aider against the Part 9 gateway and against a plain
#          Ollama install, so that the context window, the edit format and the weak model
#          are chosen deliberately rather than guessed from the model name.
# Platform: all (spark, strix, mac, nvidia)
# Minimum memory: 16 GB for the coder entry; 8 GB for the qwen3-8b entry
# Assumes: saved as .aider.model.settings.yml in your home directory, in the root of the
#          repository, or in the directory you launch Aider from. Aider's documentation
#          states that files loaded last take priority, so a repository copy overrides a
#          home-directory copy. Pass an explicit file with --model-settings-file.
#
# Every key below is documented on https://aider.chat/docs/config/adv-model-settings.html
# (read 2026-09-09). Do not invent keys here: an unknown key is silently ignored, which
# looks exactly like a setting that did not work.

# ---------------------------------------------------------------- gateway, coder model
# The name is the model string you pass to --model, including the provider prefix.
# Through the Part 9 gateway the prefix is openai/ because LiteLLM presents an
# OpenAI-compatible surface, and the rest is the gateway alias.
- name: openai/local/coder
  # Search/replace blocks. Ask a model for diffs only if it can produce them; the
  # polyglot leaderboard's edit-format column is the evidence for that decision.
  edit_format: diff
  # The repository map is what lets Aider mention files it has not been given. It costs
  # tokens on every request, so it is a context decision as much as a quality one.
  use_repo_map: true
  # Aider uses the weak model for commit messages and history summaries. Pointing it at
  # the small alias keeps the coder model loaded for the work that needs it.
  weak_model_name: openai/local/chat
  extra_params:
    # The context the gateway should allocate for this model. Raise it to match what you
    # configured behind the alias; a value larger than the server allows is refused at
    # request time, which is a clearer failure than silent truncation.
    max_tokens: 8192

# ------------------------------------------------------------------ gateway, chat model
- name: openai/local/chat
  # Whole-file edits. Slower and more expensive in tokens, and the right choice for a
  # model that cannot reliably produce search/replace blocks. Measure before choosing.
  edit_format: whole
  use_repo_map: false
  extra_params:
    max_tokens: 4096

# ------------------------------------------------------------------------ direct Ollama
# Aider's Ollama page documents the ollama_chat/ prefix as preferred over ollama/, and
# documents num_ctx under extra_params as the way to fix the context window rather than
# letting Aider size it per request. The value must not exceed what the server will give.
- name: ollama_chat/qwen3-coder:30b
  edit_format: diff
  use_repo_map: true
  extra_params:
    num_ctx: 32768

Download aider-model-settings.yml52 lines

The file is searched for in your home directory, in the root of the git repository and in the directory you launch from, and Aider’s documentation states that “Files loaded last will take priority”, so a repository copy beats a home copy. You can also name one explicitly with --model-settings-file.

✕Common mistake

Inventing a key in the settings file. An unrecognised key is ignored without complaint, so a typo produces a file that looks configured and behaves as if it is not. The documented key names are name, edit_format, weak_model_name, use_repo_map, extra_params, use_temperature, streaming, editor_model_name, editor_edit_format, reasoning_tag, system_prompt_prefix and a handful of others on the same page. Read them from the page, not from memory.

Edit formats decide whether it works

This is the part of Aider that people configure last and should configure first.

Aider does not ask the model to rewrite your file. It asks for a change, in a format, and then applies it. The format is the contract, and a model that cannot honour the contract fails in a way that looks like incompetence but is really a mismatch.

One Aider turn

1. 1Repository mapA token-budgeted summary of the repo, so the model can name files it was not given. Sized by --map-tokens; set it to zero to switch it off.
2. 2RequestSystem prompt, the map, the files you added to the chat, the conversation, and your instruction.
3. 3Reply in the edit formatSearch/replace blocks, a unified diff, or a whole file. This is where a weak model fails.
4. 4ApplyAider parses the reply and edits the files. A malformed block is rejected and Aider asks again, which costs a whole turn.
5. 5Test, if you asked for it--test-cmd runs your suite; --auto-test runs it after every change and feeds failures back.
6. 6CommitAider commits by default so every step is reversible. --no-auto-commits turns that off.

The two accented steps are the whole game. A model that produces clean diffs finishes a task in a few turns; a model that does not spends its turns being asked to try again, and the token cost of the session doubles without the outcome improving.

The documented formats are these, and their names matter because they are what you pass to --edit-format:

• whole — “the simplest possible editing format”, the model returns the entire file. It always parses. It costs a full file of output tokens per edit and it loses the rest of the file if the model gets bored, which on a long file is a real failure mode.
• diff — “asks the LLM to specify file edits as a series of search/replace blocks”. The default for capable models and the cheapest in tokens.
• diff-fenced — the same idea “but the file path is placed inside the fence”. Aider’s docs say it is “primarily used with the Gemini family of models, which often fail to conform to the fencing approach specified in the diff format”.
• udiff — “based on the widely used unified diff format, but modified and simplified”.
• editor-diff and editor-whole — “streamlined versions of the diff and whole formats, intended to be used with --editor-edit-format when using architect mode”.

The rule for local models is short: start at diff, and if the model produces malformed blocks more than occasionally, drop to whole and accept the token cost. Dropping to whole is not defeat; it is choosing a contract the model can honour. The polyglot leaderboard’s edit-format column exists precisely to record which models needed that.

⌚Version note

The Aider options page on 2026-09-09 documents --yes-always as the flag for “Always say yes to every confirmation”. A bare --yes does not appear on that page. This matters because --yes-always is what a script needs, and a flag that does not exist fails at the worst moment, which is inside an unattended run. Check flags against the options page for the version you have installed rather than copying them from a blog post.

A non-interactive run, which is what the lab needs

Aider will run one instruction and exit, which is how the lab later in this part drives it alongside five other tools.

RunnableAll tracks

one instruction, tests after every change, no commits

aider --model openai/local/coder \
  --message "Make the failing tests in task-tests.py pass. Do not change the tests." \
  --test-cmd "python3 -m pytest -q task-tests.py" \
  --auto-test \
  --no-auto-commits \
  --yes-always \
  task-app.py

Every option there is on the options page: --message “Specify a single message to send the LLM, process reply then exit (disables chat mode)”, --test-cmd “Specify command to run tests”, --auto-test “Enable/disable automatic testing after changes”, --no-auto-commits as the negative of the auto-commits toggle, and --yes-always. The bare filename at the end adds that file to the chat.

--auto-test is the option that turns Aider from an editor into an agent. Without it, the model proposes an edit and stops. With it, the test output comes back into the conversation and the loop continues until the tests pass or you run out of patience.

⛨Security

--yes-always disables the confirmation prompts, including the one before Aider runs your test command. That is fine when the test command is yours and the repository is yours. It is not fine in a directory you did not write, and it is exactly the setting the sandbox lab in this part exists to contain. Read the second lab before you use this flag on anything you did not create.

The benchmark Aider brings with it

The polyglot benchmark is the reason Aider is worth more to this course than its size suggests: it is a reproducible, execution-graded measurement you can run on your own weights.

It is described as “225 challenging Exercism coding exercises across C++, Go, Java, JavaScript, Python, and Rust”. The selection method is the interesting part, and it is stated in the announcement: from Exercism’s 697 problems the authors kept “the 225 problems that were solved by 3 or fewer models” out of seven they tested. The benchmark is deliberately made of the exercises that were already hard, which is why absolute scores on it are low and why comparing it with an older, easier benchmark is meaningless.

Running it is documented, and the documentation leads with a warning you should not skip: “The benchmark is intended to be run inside a docker container. This is because the benchmarking harness will be taking code written by an LLM and executing it without any human review or supervision!”

Fragment — not complete on its own

./benchmark/benchmark.py <run-name> \
  --model openai/local/coder \
  --edit-format diff \
  --threads 1 \
  --exercises-dir polyglot-benchmark

That invocation is from the harness README and is shown as a fragment because it must be run from a clone of the Aider repository with the exercises repository fetched beside it, inside the container, with your endpoint variables set. Results come back through ./benchmark/benchmark.py --stats <directory>, which reports the pass rate as “the percent of the tasks which had all tests passing”, along with malformed-response and error counts.

▷Exercise

Run it once with --edit-format diff and once with --edit-format whole against the same local model, on a subset if a full run is too long for your machine. The pass rate may barely move. The malformed-response count will move a lot, and that difference is the single most useful number you can have about a local model before you let an agent loose with it.

Reading the leaderboard critically

The public leaderboard is genuinely useful and routinely misread. Three things to hold in mind, all of them checkable on the page itself.

It carries a date. On 2026-09-09 the page stated it was last updated on 20 November 2025. Everything on it is therefore about ten months old, in a field where ten months is two model generations.

Almost none of it was run the way you will run it. Each row’s command names a hosted provider. None of them states a quantisation. A 4-bit GGUF of the same weights on your machine is a different artefact, and the gap between the two is exactly what Part 16 taught you to measure.

The second column is the one for local models. Percent correct tells you whether the model can solve the exercise. Percent using the correct edit format tells you whether it can talk to Aider, and that is the property that decides whether the tool is usable at all on your hardware.

The honest use of the leaderboard is as a shortlist generator. It tells you which open-weight families are worth downloading. Your own run of the same benchmark, on your own quantisation, tells you what to configure.

Inspect an edit failure before increasing model size

Aider must identify the relevant files, ask for an edit in the chosen format and apply that edit to the current repository. A failed patch can result from a stale context, an unsuitable edit format or a model that cannot follow the format. Preserve the raw proposed edit and the application error before retrying.

Use a disposable checkout at a recorded commit. Give the agent a bounded task and run the independent test after it finishes. Inspect the diff for unrelated changes and edits to the tests themselves. Reset the experimental checkout to the same starting state between candidates while retaining each result outside it.

Compare context size and repository-map settings as separate treatments. More context may help locate a dependency but also increase prefill and distract a small model. Keep model, sampling and task fixed while changing those settings. The practical success criterion is a correct applied change with an understandable diff and acceptable operator effort, not simply a response that resembles a patch.

Recap

Aider reaches your gateway with OPENAI_API_BASE, OPENAI_API_KEY and a model name prefixed openai/, or reaches Ollama with OLLAMA_API_BASE and the ollama_chat/ prefix. The context window is the setting that silently ruins everything, and it must be raised in the server and pinned per model in .aider.model.settings.yml. The edit format is the contract between model and tool: start at diff, fall back to whole when the blocks come back malformed, and treat that fallback as information about the model rather than a defeat. --message with --test-cmd and --auto-test turns it into an agent that keeps going until the suite passes, which is exactly what the lab needs. And the polyglot benchmark ships with the tool, runs in a container for good reasons, and is worth more run once on your own weights than read a hundred times on somebody else’s leaderboard.

Check your understanding

Question 1. Why must a model name passed to Aider carry a provider prefix such as openai/ or ollama_chat/?

• It is cosmetic and helps readability
• The prefix selects the routing path; without it the bare name is resolved against a table of known hosted models and will not reach your endpoint
• It sets the context window
• It chooses the edit format

Show the answer and why

Answer: The prefix selects the routing path; without it the bare name is resolved against a table of known hosted models and will not reach your endpoint

Aider routes by prefix. Through the Part 9 gateway the prefix is openai/ because LiteLLM presents an OpenAI-compatible surface, and the remainder is your alias, so the full string reads openai/local/coder.

Question 2. A local model in Aider forgets the beginning of the conversation and produces increasingly irrelevant edits. What should you check first?

• The edit format
• The context window on the serving side and the value Aider requests per model, since a small context is discarded silently rather than reported
• The repository map size
• The temperature

Show the answer and why

Answer: The context window on the serving side and the value Aider requests per model, since a small context is discarded silently rather than reported

Aider's Ollama page states that Ollama defaults to a 2k window and silently discards excess context. Behind a gateway the same trap lives one layer down in the engine's ctx-size. Nothing errors, which is what makes it worth checking first.

Question 3. Your model produces malformed search/replace blocks about a third of the time. Which response is most defensible?

• Increase the temperature so it varies its output
• Switch to the whole edit format, accept the extra output tokens, and record that the model could not hold the diff contract
• Give up on the model entirely
• Turn off the repository map

Show the answer and why

Answer: Switch to the whole edit format, accept the extra output tokens, and record that the model could not hold the diff contract

The whole format always parses, at the cost of a full file of output tokens per edit. Choosing a contract the model can honour is a configuration decision, and the leaderboard has a column for exactly this because it is a normal property of smaller models.

Question 4. Which statements about the Aider polyglot leaderboard are true? Select all that apply.

• It carries a last-updated date, which was 20 November 2025 when read on 2026-09-09
• Its exercises were selected for difficulty, being those solved by three or fewer of seven tested models
• Rows state the quantisation used
• The edit-format column matters more than the correctness column when judging a local model for agent work

Show the answer and why

Answer: It carries a last-updated date, which was 20 November 2025 when read on 2026-09-09, Its exercises were selected for difficulty, being those solved by three or fewer of seven tested models, The edit-format column matters more than the correctness column when judging a local model for agent work

No row states a quantisation, which is the single biggest reason a leaderboard number does not transfer to a GGUF on your machine. The other three are on the pages and are what make the leaderboard useful as a shortlist rather than as a prediction.

Question 5. Why does the Aider benchmark documentation insist the harness runs inside a container?

• To make it faster
• Because the harness executes code written by a model without human review, so it must not run against your own filesystem
• Because the exercises need a specific Linux distribution
• Because the container includes the model weights

Show the answer and why

Answer: Because the harness executes code written by a model without human review, so it must not run against your own filesystem

The README says so directly. It is the same principle the sandbox lab in this part builds on: code you did not read, executed automatically, needs a boundary around it that is not your home directory.

Check my answersClear