Agent Frameworks Compared

By the end of this lesson you will be able to apply one test to any agent framework and get a useful answer in an afternoon: does it take a base URL and a model name, and how much does it fight you about it. You will also be able to say what a framework buys you over the hundred-line loop from Part 24, which of those things you actually need, and which single framework in this survey cannot talk to an OpenAI-compatible server without something in front of it.

The one criterion, first

Every framework comparison you will read ranks features. Features are not the constraint here. The constraint is that your model is on your own machine behind an OpenAI-compatible endpoint, and a great many agent frameworks were designed by people whose model was always a hosted one behind a key.

So the test is this. Can you point it at a base URL and a model name, using one documented constructor argument, without a shim, a fork or a monkey patch? Everything else is negotiable. A framework that fails this test will cost you a weekend before it runs at all, and will cost you another one every time it is upgraded.

∴Research note

The table below is the result of running that test against the documentation of each project on 2026-09-09, not against a benchmark. Each row says what the project’s own pages show. None of these frameworks is pinned in the course’s version file, so where a page states a version this lesson names it and where it does not, it says so. Re-run the test yourself before trusting any row: this is the fastest-moving corner of the subject in the whole course.

The base-URL test, nine ways

Framework	Verdict	The mechanism its documentation shows
LangGraph (with LangChain)	Native	ChatOpenAI(base_url=…, api_key=…)
LlamaIndex	Native	OpenAILike(api_base=…, is_chat_model=True)
Pydantic AI	Native	OpenAIChatModel(alias, provider=OpenAIProvider(base_url=…))
smolagents	Native	OpenAIModel(model_id=…, api_base=…, api_key=…)
OpenAI Agents SDK	Native, with a switch	AsyncOpenAI(base_url=…) plus set_default_openai_api("chat_completions")
Google ADK	Native, through LiteLLM	LiteLlm(model="hosted_vllm/…", api_base=…)
CrewAI	Native	LLM(model=…, base_url=…)
AutoGen	Documented, untested by the project	OpenAIChatCompletionClient with model_info
Claude Agent SDK	Needs a proxy	ANTHROPIC_BASE_URL at a gateway speaking the Anthropic API

Seven of the nine take your gateway directly. That is a better result than the folklore suggests, and it is a recent one: the OpenAI-compatible shape has become the lingua franca of local serving, and the frameworks followed. Here is each mechanism, as its own documentation writes it.

LangGraph has no model layer of its own. Its overview describes it as “a low-level orchestration framework and runtime for building, managing, and deploying long-running, stateful agents”, and says plainly that “you don’t need to use LangChain to use LangGraph” — LangChain supplies the model and tool abstractions. So the base URL goes into LangChain’s OpenAI chat model, which the integration page instantiates with base_url and api_key:

Fragment — not complete on its own

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="local/answer", base_url="http://127.0.0.1:4000/v1",
                 api_key="your-gateway-key")

LlamaIndex describes itself as “the leading framework for building LLM-powered agents over your data with LLMs and workflows”. Its vLLM example is blunt about which class to use for a served model: the vLLM client module “is a client for vllm.entrypoints.api_server which is only a demo”, and for production OpenAI-compatible servers you should use OpenAILike, which the API reference calls “a thin wrapper around the OpenAI model that makes it compatible with 3rd party tools that provide an openai-compatible api”.

Fragment — not complete on its own

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(model="local/answer", api_base="http://127.0.0.1:4000/v1",
                 api_key="your-gateway-key", context_window=32768,
                 is_chat_model=True, is_function_calling_model=True)

The two boolean arguments matter more than they look. is_chat_model decides whether the library talks to the chat endpoint or the completion endpoint, and is_function_calling_model decides whether it will attempt tool calls at all. Leave the second one at its default and your agent will quietly behave like a chatbot.

Pydantic AI calls itself “the Python AI SDK: a typed, extensible agent loop with every model a string swap away”, and documents the compatible-provider case directly:

Fragment — not complete on its own

from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel(
    "local/answer",
    provider=OpenAIProvider(base_url="http://127.0.0.1:4000/v1", api_key="your-gateway-key"),
)

Its documentation is also careful about which OpenAI API is in play: the bare openai: prefix resolves to the Responses API, and OpenAIChatModel is the Chat Completions one. Local servers speak Chat Completions, so name the class rather than relying on a prefix.

smolagents is “an open-source Python library designed to make it extremely easy to build and run agents using just a few lines of code”, and its models reference introduces OpenAIModel with the sentence “This class lets you call any OpenAIServer compatible model. Here’s how you can set it (you can customise the api_base url to point to another server)”:

Fragment — not complete on its own

from smolagents import OpenAIModel, ToolCallingAgent

model = OpenAIModel(model_id="local/answer", api_base="http://127.0.0.1:4000/v1",
                    api_key="your-gateway-key")
agent = ToolCallingAgent(tools=[], model=model, max_steps=8)

The documentation read here is version 1.26.0. It also ships VLLMModel and MLXModel for running weights in-process, which is a different thing from talking to your gateway and is worth ignoring while you are testing the endpoint.

The OpenAI Agents SDK takes a base URL, but only after you turn something off. Its models page says “The SDK uses the Responses API by default, but many other LLM providers still do not support it”, which is exactly the failure you would otherwise spend an evening on: a 404 from an endpoint that is working perfectly well.

Fragment — not complete on its own

from agents import Agent, AsyncOpenAI, OpenAIChatCompletionsModel, set_tracing_disabled

set_tracing_disabled(disabled=True)
client = AsyncOpenAI(api_key="your-gateway-key", base_url="http://127.0.0.1:4000/v1")
model = OpenAIChatCompletionsModel(model="local/answer", openai_client=client)
agent = Agent(name="Helping Agent", instructions="You are a Helping Agent", model=model)

Tracing is the second thing to know about: the page explains that a 401 during a local run happens “because traces are uploaded to OpenAI servers, and you don’t have an OpenAI API key”. On a private machine that upload is not a nuisance, it is a disclosure, so disable it deliberately rather than because an error told you to.

Google ADK reaches local endpoints through a LiteLLM wrapper. Its models page lists LiteLlm alongside Ollama, vLLM and LiteRT-LM hosting, and its vLLM page shows the wrapper taking api_base with a model name in LiteLLM’s hosted_vllm/ form:

Fragment — not complete on its own

from google.adk.agents import LlmAgent
from google.adk.models.lite_llm import LiteLlm

agent_vllm = LlmAgent(
    model=LiteLlm(model="hosted_vllm/local/answer", api_base="http://127.0.0.1:4000/v1"),
    name="vllm_agent",
    instruction="You are a helpful assistant running on a self-hosted vLLM endpoint.",
)

CrewAI documents the same idea in one class, with an Ollama example alongside the gateway form:

Fragment — not complete on its own

from crewai import LLM

llm = LLM(model="openai/local/answer", base_url="http://127.0.0.1:4000/v1",
          api_key="your-gateway-key")

AutoGen is the row to read carefully, for two reasons. The first is that the name covers two projects: AG2’s own documentation states that “autogen and ag2 are aliases for the same PyPI package” (its page footer dated 2026-06-26, read 2026-09-09), while Microsoft’s AutoGen is a separate set of documentation with its own model clients. The second is that Microsoft’s model tutorial adds a caveat none of the others do: “You can use this client with models hosted on OpenAI-compatible endpoints, however, we have not tested this functionality.” That is not a refusal, and it is not a promise. It is a project telling you which of you is doing the testing. Its model_info dictionary, which declares vision, function_calling, json_output, family and structured_output, is where an unknown local model’s capabilities have to be asserted by hand.

⌚Version note

Every claim in this section is dated 2026-09-09 and several of these projects moved documentation domains during 2026: LangGraph’s pages redirect from the old site to docs.langchain.com, LlamaIndex’s to developers.llamaindex.ai, Pydantic AI’s to pydantic.dev/docs/ai, and Google ADK’s to adk.dev. If a URL in this page’s source list redirects, follow it rather than assuming the project is gone.

The one that needs a proxy, and why

The Claude Agent SDK is the exception, and the reason is instructive rather than political. Its overview describes it as giving you “the same tools, agent loop, and context management that power Claude Code, programmable in Python and TypeScript”. That loop speaks the Anthropic Messages API, not the OpenAI Chat Completions API, so an OpenAI-compatible server is not a thing it can address at all. What exists instead is a gateway mechanism: ANTHROPIC_BASE_URL “is the variable that points Claude Code at the gateway”, and any gateway “that exposes a supported API format works”.

Why one row of the table is different

a request travels down; the shape it is written in changes once

1. Seven frameworksCompose an OpenAI chat-completions request with a tools arraybase_url is enough
2. The Claude Agent SDKComposes an Anthropic Messages request; there is no OpenAI mode to switch toa translator is required
3. A translating gatewayLiteLLM's Anthropic-shaped endpoint from Part 9, or llama-server's, as Part 25 configuresthis is the proxy
4. Your enginellama-server, vLLM, SGLang or MLX, speaking whatever it speaks
5. The weightsUnchanged by any of the above

Nothing here is a judgement about the SDK. It is a statement about which wire format the loop was written in, and where the translation therefore has to happen.

Two things are worth saying plainly about that arrangement. Anthropic’s own gateway page states that Anthropic “doesn’t endorse, maintain, or audit third-party gateway products, and doesn’t support routing Claude Code to non-Claude models through any gateway”, so this is a configuration you own the consequences of. And a gateway “becomes infrastructure your organization operates”: when the SDK adds a capability, a proxy that does not forward it breaks the corresponding feature. Part 25 configures this path properly and reports what works; this lesson’s job is only to explain why it is the odd row out.

What a framework adds over the loop you wrote

Part 24’s minimal-agent.py is about two hundred lines and it works. So the question is not “is a framework better”, it is “which of these five things do I need enough to accept a dependency for”.

What you get, in the order you usually come to need it

1. 1Typed tools from ordinary functionsA decorator turns a Python function into a JSON Schema, reading the signature and the docstring. This is the one that saves the most tedium, because hand-written schemas drift from the functions they describe.
2. 2Typed outputThe final answer arrives as a validated object rather than a string you parse hopefully. Part 10 built this by hand with a JSON schema and Pydantic; a framework makes it the default.
3. 3State that survives a restartA checkpoint after every step, a thread identifier to resume from, and the ability to inspect or rewind. Needed when a run lasts longer than a terminal session.
4. 4Streaming that is more than tokensSeparate streams for tokens, state updates and tool events, so a user interface can show what the agent is doing rather than only what it is saying.
5. 5Composition of several agentsOne agent calling another as if it were a tool, with its own memory and its own tool set. You can write this yourself; the value is that the framework has already decided how state passes between them.

Nothing in this list is impossible in your own loop. Each is a place where a framework has made a decision you would otherwise make badly the first time.

State and persistence

LangGraph is the clearest example because persistence is its subject rather than a feature. Its documentation says checkpointers “persist a thread’s graph state as checkpoints” and describes them as “short-term, thread-scoped memory”, with a separate store for “long-term, cross-thread memory”. The library ships an in-memory saver, a SQLite saver described as “local file-based storage for development”, and a PostgreSQL saver. The use cases it lists are the ones that matter for a local agent that runs for an hour: “conversation continuity, human-in-the-loop, time travel, and fault tolerance”.

Time travel is the underrated one. If every step is checkpointed, you can rewind to the step before the agent went wrong, change one thing, and run forward again. In your own loop, that is a re-run from the beginning, at full token cost, with a different random seed.

Streaming

LangGraph’s stream modes are a good vocabulary for what streaming means in an agent, because they name the things separately: values gives the “full state after each step”, updates gives “state updates after each step”, messages gives “2-tuples of (LLM token, metadata) from LLM calls”, custom carries whatever a node chooses to emit, and tasks gives “task start/finish events with results and errors”, which requires a checkpointer.

The distinction to hold onto is that token streaming is a user-interface feature and event streaming is an operations feature. On local hardware the second is the one that earns its place, because a slow turn and a stuck turn look identical until something tells you which tool is running.

Typed tools and typed output

Pydantic AI’s pitch is this: “Structured outputs, typed dependency injection, typed tools: your IDE, type checker, and coding agent all know what your agent returns.” In practice the mechanism is small and worth copying even if you use no framework at all. A tool is a function; the decorator reads its signature for the schema and its docstring for the descriptions, and the documentation says Pydantic AI “extracts the docstring from functions and (thanks to griffe) extracts parameter descriptions from the docstring and adds them to the schema”.

Fragment — not complete on its own

@researcher.tool
def search_documents(ctx: RunContext[Deps], query: str, limit: int = 5) -> str:
    """Search the document collection for passages matching a query.

    Args:
        query: Search terms, as words rather than a question.
        limit: How many passages to return, 1 to 6.
    """

Part 24 made the point that a tool’s description is the model’s entire interface to it. Generating that description from the docstring is what stops the two from drifting apart after the third refactor.

The output side is the same trick pointed the other way. An agent declared with an output_type returns a validated object, and a run can be bounded with UsageLimits, which takes a request_limit among others. That limit is a stopping condition of exactly the kind Part 24’s loop implements by hand, and having it in the constructor rather than in your own while is the difference between remembering it and not.

Code agents, and why this course does not start there

smolagents draws a distinction the others mostly do not. Its CodeAgent “generates tool calls as Python code snippets”, which the guided tour argues is “highly expressive” and allows “complex logic and control flow”, while its ToolCallingAgent “writes tool calls as structured JSON”, which is “reliable” and “safe: arguments are strictly validated, no risk of arbitrary code running”. Its own guidance is to use the code agent when “you need reasoning, chaining, or dynamic composition” and the tool-calling agent when “you have simple, atomic tools” and “want high reliability and clear validation”.

⛨Security

A code agent executes model-written Python. The smolagents documentation is explicit that the default local interpreter restricts imports and that the warning applies regardless: “The LLM can generate arbitrary code that will then be executed: do not add any unsafe imports!” It supports sandboxed execution through several hosted services or Docker. This course uses the tool-calling shape in its own scripts, because a small local model generating control flow is exactly the case where a mistake and an attack look identical. Run a code agent, when you run one, in the container Part 25 builds.

Choosing by the shape of the problem

Having passed the base-URL test, the frameworks differ less than their marketing does. What actually decides is the shape of the thing you are building.

If the problem is…	The shape you want	Where to look first
A long-running process that must survive a restart	A graph with checkpoints and threads	LangGraph
Questions over your own documents, with retrieval as the centre	An index-first framework with agents on top	LlamaIndex
A typed pipeline where the answer must validate	Typed tools and a typed output	Pydantic AI
One agent, three tools, running today	The smallest thing that works	smolagents, or Part 24’s loop
A team of role-playing agents with a written brief	Agents as personas with tasks	CrewAI
An existing Google or OpenAI codebase you are extending	Their own SDK, plus the base-URL switch	ADK, OpenAI Agents SDK
Claude Code’s exact loop and tools, on your own model	The SDK behind a translating gateway	Claude Agent SDK, and Part 25

✕Common mistake

Choosing a framework before writing the loop. The failure this causes is not a bad choice of framework; it is not being able to tell whether a disappointing agent is the model’s fault or the scaffold’s. Part 24’s loop is the control condition, and this part’s reality check measures exactly that. Write the loop, measure it, and adopt a framework when you can name the thing it does that you are doing badly.

The honest summary is that on a local endpoint the framework is rarely what decides whether your agent works. The model’s tool-call reliability decides it, the tool descriptions decide it, and the context budget decides it, and all three are Part 24’s subject. What a framework decides is how pleasant the code is to change in three months, which matters, but not on the first afternoon.

Evaluate the framework’s state and failure contracts

A framework’s useful contribution is often state persistence, typed tool handling, tracing or recovery rather than the loop itself. Choose a task that requires the feature you need, and compare it with the minimal loop under the same model and budget.

Inject a tool exception and restart at a known checkpoint. Inspect whether completed side effects are repeated, whether messages retain their call identifiers and whether cancellation propagates. A saved conversation alone may not record enough external state for a safe resume. Test the application’s operation identities independently of framework abstractions.

Record the resolved prompt and tool schemas where available. Framework defaults can change the context and explain a quality difference attributed to orchestration. Include additional model calls, tokens and dependencies in the comparison. A framework is a good fit when it supplies a needed, tested contract and makes failures easier to understand. A larger feature surface does not automatically improve a bounded local task, and API compatibility should be demonstrated before comparing the framework’s higher-level behaviour.

Recap

The base-URL test is one question: does the framework take a base URL and a model name from a documented constructor argument. Seven of the nine surveyed here do, each with a mechanism its own documentation shows: base_url on LangChain’s chat model under LangGraph, api_base on LlamaIndex’s OpenAILike, an OpenAIProvider in Pydantic AI, api_base on smolagents’ OpenAIModel, an AsyncOpenAI client plus a switch to Chat Completions in the OpenAI Agents SDK, a LiteLLM wrapper in Google ADK, and base_url on CrewAI’s LLM. AutoGen documents the same shape while stating it has not tested it. The Claude Agent SDK is the exception: it speaks the Anthropic Messages API, so it reaches a local model only through a translating gateway pointed at by ANTHROPIC_BASE_URL, which Part 25 configures. Over Part 24’s loop, a framework adds typed tools generated from function signatures, typed and validated output, checkpointed state you can resume and rewind, event streams that tell you what the agent is doing, and a decided-in-advance way for agents to call each other. None of it changes whether your model can emit a valid tool call, which is still the thing that decides whether any of this works.

Check your understanding

Question 1. You point a framework at your gateway and every request comes back 404, although curl against the same base URL works. Which framework is this most likely to be, and why?

• LlamaIndex, because OpenAILike defaults to the completion endpoint
• The OpenAI Agents SDK, because it uses the Responses API by default and most local servers implement Chat Completions
• CrewAI, because it requires an Ollama-style model prefix
• LangGraph, because it has no model layer of its own

Show the answer and why

Answer: The OpenAI Agents SDK, because it uses the Responses API by default and most local servers implement Chat Completions

The SDK's own models page says it "uses the Responses API by default, but many other LLM providers still do not support it", and names the two fixes: call set_default_openai_api("chat_completions") or instantiate OpenAIChatCompletionsModel. A 404 from a healthy server is nearly always a request sent to a path the server does not implement.

Question 2. Which of these are true of the Claude Agent SDK on a local model? Select all that apply.

• It composes requests in the Anthropic Messages shape rather than the OpenAI chat-completions shape
• ANTHROPIC_BASE_URL points it at a gateway
• Anthropic supports routing it to non-Claude models through a gateway
• A gateway that does not forward a new capability will break the corresponding feature

Show the answer and why

Answer: It composes requests in the Anthropic Messages shape rather than the OpenAI chat-completions shape, ANTHROPIC_BASE_URL points it at a gateway, A gateway that does not forward a new capability will break the corresponding feature

The first, second and fourth are stated in Anthropic's own gateway documentation, including that the gateway "becomes infrastructure your organization operates" and must be kept current. The third is the opposite of what the page says: it "doesn't support routing Claude Code to non-Claude models through any gateway". The mechanism exists; the support does not.

Question 3. You set is_function_calling_model to its default when constructing LlamaIndex's OpenAILike, and your agent never calls a tool. What happened?

• The model is too small to emit tool calls
• The library was not told the endpoint supports tool calling, so it did not attempt any
• The gateway stripped the tools array
• The context window was too small for the schemas

Show the answer and why

Answer: The library was not told the endpoint supports tool calling, so it did not attempt any

OpenAILike takes is_chat_model and is_function_calling_model precisely because an arbitrary compatible server cannot be interrogated about its capabilities. Both are assertions you make. This is the same class of problem as AutoGen's model_info dictionary, where vision, function_calling, json_output and family have to be declared by hand for a model the library does not recognise.

Question 4. A small local model is driving a smolagents CodeAgent with a filesystem tool, unsandboxed, and it writes a snippet that deletes a directory. Where should the control have been?

• In the system prompt, forbidding destructive operations
• In the choice of agent type and the execution environment: a tool-calling agent validates arguments against a schema, and a code agent belongs in a container
• In a larger model, which would not have made the mistake
• In the framework, which should refuse dangerous code

Show the answer and why

Answer: In the choice of agent type and the execution environment: a tool-calling agent validates arguments against a schema, and a code agent belongs in a container

smolagents documents the trade-off itself: the tool-calling agent has "arguments strictly validated, no risk of arbitrary code running", while the code agent needs a sandbox and carries the warning that the model "can generate arbitrary code that will then be executed". A prompt is a request to a token predictor; the agent type and the container are properties of the system.

Question 5. Why does this lesson insist on the base-URL test before any feature comparison?

• Because features do not matter in agent frameworks
• Because a framework that cannot address a local endpoint through a documented argument costs you a rewrite before you can evaluate anything else, and will cost you another at every upgrade
• Because all these frameworks have the same features
• Because local endpoints are slower than hosted ones

Show the answer and why

Answer: Because a framework that cannot address a local endpoint through a documented argument costs you a rewrite before you can evaluate anything else, and will cost you another at every upgrade

It is a filter, not a ranking. Features are worth comparing only among the frameworks you can actually run on your own hardware, and the shim that makes an incompatible framework work is code you own, that nobody tests, and that breaks on upgrade.

Check my answersClear