Skip to main content
The AI gateway is the LLM proxy on the Agent Controller. A session gets ANTHROPIC_BASE_URL (Claude Code) or OPENAI_BASE_URL (Codex) pointing at it, plus a session token. No executor, workspace or harness ever holds a provider key.

​
What passes through it

  • Every turn of an agent on Claude Code, Codex or OpenCode.
  • xpander’s own model calls: thread titles, context compaction, evaluation judges, and the gateway model that filters entries ahead of the agent’s model.
  • Guardrail moderation, when enabled.
For each call, the proxy resolves the credential, applies the model policy and budget, records the call under the person who asked, and forwards it.

​
On a self-hosted install

The controller forwards calls to the LLM Gateway in the cluster:
  • The gateway is always on. It applies allowed models, provider keys, fallbacks and budgets, and logs every call.
  • Every provider your agents use needs a key, saved in the organization or set on the install (secrets.static). Otherwise the call is refused with a message naming the missing key.
  • New runs wait for a healthy gateway. See LLM Gateway on a self-hosted install.
  • Preflight refuses an install with no provider for model calls: an organization default, a gateway provider, or a per-provider in-cluster endpoint must exist.

​
Where it is configured

The organization’s default provider sets a new agent’s harness: Anthropic gives Claude Code, Amazon Bedrock with an Anthropic model gives Claude Code on Bedrock, OpenAI gives Codex, any other provider gives OpenCode on that model.

​
Any model on any harness

The gateway translates between each harness’s API and each provider’s API. So Claude Code, Codex and OpenCode can each run any model your organization allows, not only their own vendor’s models. On an Air-Gapped install, all three harnesses can run the model of your own provider (for example vLLM on your own GPUs).
  • Pick the model in the composer’s Harness and model picker. See Harness and model.
  • A harness lists its own vendor’s models, plus models from other vendors that passed xpander’s test on that harness: chat, a tool call, streaming, long context and images (when the model accepts images). Untested pairs are not listed.
  • A model from your own provider is listed for a harness after an admin runs Test model on it for that harness and it passes.
  • Each turn is recorded in Platform > AI governance > Logs with the model that ran it.
  • A personal Claude or ChatGPT plan applies only to its own vendor’s models. A model from another vendor runs on the organization’s providers.

​
Routing through a gateway you already run

Each provider’s upstream is an environment variable on the Agent Controller, so your own AI gateway can sit between xpander and the vendor: The controller egresses to api.anthropic.com by default. With an override, the override is the only host it needs. xpander applies its policy, budget and record before the call leaves. Your gateway’s other clients stay outside xpander’s controls.

​
Verify

1

Check the Agent Controller

2

Check the LLM Gateway pods

A healthy gateway shows two llm-gw pods (one on a single-node cluster installed with install.sh), each 1/1 and Running.
3

Check the product

The Locations tab of Platform > Agents > Runtime shows the installation as Connected. A model call appears in the task’s usage and in Platform > AI governance > Logs.

​
Next steps

Set up providers and keys

On a self-hosted install.

Run a local model air-gapped

Local LLM inference.

Manage LLM keys on Hybrid

Where provider keys live.