Skip to content
Local LLM Course

Ollama, LM Studio and the Desktop Experience

Most people who run a model locally have already installed Ollama or LM Studio, usually before reading anything. Both are good. Both also make decisions on your behalf and do not announce them: how much context a conversation gets, how long a model stays in memory, where thirty gigabytes of weights went, which of your GPUs is being used. Part 6 built the engine by hand so that none of that would be mysterious. This part puts the convenience back on top, with the lid open.

The order is deliberate. You learned to drive llama-server first, so when Ollama’s defaults surprise you, you already know what a context length is and what it costs. A reader who starts here instead learns the tool and not the subject, and is stuck the first time the tool does something the documentation does not explain.

What you will be able to do

Section titled “What you will be able to do”

By the end of this part you should be able to:

  • install Ollama on your track, tell whether it found your GPU, pull and run models, write a Modelfile that changes a model’s context length and system message, and read the defaults for context and keep-alive rather than inheriting them;
  • decide when LM Studio is the right tool, drive it from the lms command line without opening the application, run it headless as a server, and say which of its two engines a given model is using on a Mac;
  • put a chat front-end in front of any OpenAI-compatible server, state Open WebUI’s licence condition accurately before you deploy it for other people, and name two alternatives and their licences;
  • keep one copy of each model on disk, shared between llama.cpp, Ollama and LM Studio, with the quantisation, source and date recorded, and prune the copies you no longer run;
  • stand up a chat service on your home network with TLS, accounts and a backup, reachable from every device in the house and from nowhere else;
  • measure what a default context length does to a long conversation, price the memory that a larger one costs, and choose a number for your own household with the arithmetic in front of you.

What you need first

Section titled “What you need first”

Part 5 established which track you are on and how much memory the graphics processor can actually see, which is the number every fit decision in this part uses. Part 6 supplies llama-server, GGUF files, the -c flag and the habit of watching a server’s logs; the lab here will happily use that server instead of Ollama, and the reality check measures both. Part 4 supplies the key-value cache arithmetic, which is the whole of the second half of the reality check, and the ~/models library that the storage lesson extends.

The lab and the reality check both run on the 8 GB tier with a 4B-class model. Nothing in this part needs a second machine.

How to work through it

Section titled “How to work through it”

Read the Ollama and LM Studio lessons in either order; they are independent, and if you only ever use one of the two, read that one carefully and skim the other for the comparison. The front-ends lesson depends on neither and is worth reading before the lab, because the lab deploys the thing it describes and the licence question is easier to think about before the container is running.

Do the storage lesson before you download anything else. It is the least exciting page in this part and the one that saves the most disk: by Part 9 you will have several tools that each want their own copy of the same weights, and the layout that stops that is a decision you make once.

Then the lab, which is the first thing in this course you can show someone else, and then the reality check, which will probably change the number you put in the lab’s configuration. Doing them in that order is intentional: you deploy with a default, measure why it is wrong for you, and change it, which is how every service you run will actually go.

Part7 of 27Level2 — Local OperatorPages6Estimated time3.5 hours

0 / 6 lessons in this part completed