# Gateway

Purpose: the record of what your local model gateway is, for the parts of this course that
come after Part 9 and for you in six months. Save this file as `GATEWAY.md` beside your
`compose.yaml`, fill in every angle-bracketed blank, and update it whenever you change the
stack. Platform: all. Minimum memory: 8 GB. Assumes: the gateway from Part 9's project is
running. Nothing in this file is a secret: keys live in `.env`, which is not committed.

---

## What this machine serves

| Field | Value |
| --- | --- |
| Machine name | `<the name you call it>` |
| Track | `<S, X, M or N>` |
| Accelerator and memory | `<chip and total memory>` |
| Operating system | `<name and version>` |
| Gateway address | `http://127.0.0.1:<port>` |
| Started by | `<docker compose up -d, or bash start-gateway-native.sh>` |
| Last changed | `<date>` |

## Endpoints

| Path | Shape | What it is for |
| --- | --- | --- |
| `/v1/chat/completions` | OpenAI | Every chat and agent client in Parts 10, 24, 25 and 26 |
| `/v1/completions` | OpenAI | Fill-in-the-middle and raw completion |
| `/v1/embeddings` | OpenAI | The retrieval work in Part 10 |
| `/v1/models` | OpenAI | What a client asks to discover the aliases below |
| `/v1/messages` | Anthropic | Clients that speak the Anthropic message format |
| `/health/liveliness` | Health | The process is up |
| `/health/readiness` | Health | The worker is ready to accept traffic |
| `/health` | Health | A real request against each configured model |

## Model aliases

These names are the contract. Later parts of the course use them, and changing one means
changing everything that refers to it, so add rather than rename.

| Alias | Model and quantisation | Context | Engine | Notes |
| --- | --- | --- | --- | --- |
| `local/chat` | `<model, quantisation>` | `<tokens>` | `<engine and version>` | General conversation |
| `local/coder` | `<model, quantisation>` | `<tokens>` | `<engine and version>` | Code; used from Part 25 |
| `local/embed` | `<model, quantisation>` | `<tokens>` | `<engine and version>` | Retrieval; used from Part 10 |
| `<any alias you added>` | | | | |

Licences of the models above, from the course model reference:
`<one line per model>`

## Keys

| Key | Who holds it | What it may reach | Created |
| --- | --- | --- | --- |
| Master key | You, in `.env` only | Everything, including key creation | `<date>` |
| `<a virtual key name>` | `<which application>` | `<which aliases>` | `<date>` |

The master key is not an application credential. Applications get a virtual key with a model
list and a budget, so that revoking one does not disturb the others.

## Routing and fallbacks

| Alias | Retries | Falls back to | Why |
| --- | --- | --- | --- |
| `local/coder` | `<n>` | `<alias>` | `<one sentence>` |
| `local/chat` | `<n>` | `<alias>` | `<one sentence>` |

Model swapping: `<which models share the accelerator and are swapped, and which stay resident>`

Idle unload: `<the ttl values you chose, and why>`

## Logging

| What | Where | Retention |
| --- | --- | --- |
| Router request log | `<path or docker compose logs gateway>` | `<how long you keep it>` |
| Engine logs | `<path or docker compose logs swap>` | `<how long you keep it>` |
| Usage and spend records | `<the database, or "not enabled on this path">` | `<how long you keep it>` |

Prompt and response text: `<recorded, or not recorded, and the setting that decides it>`

## Operating notes

Starting from cold: `<the command, and how long the first request takes while a model loads>`

Changing a model: `<edit llama-swap.yaml, then the command that reloads it>`

What breaks first under load: `<from your own measurements in the Part 9 lab>`

Known limitations on this machine: `<for example, which models do not fit, or which track
features are unavailable>`

## Verification

Last run of `check-gateway.sh`: `<date>`, `<n>` passed, `<n>` failed.

Anything that failed, and why it is acceptable: `<one line each, or "nothing">`