> ## Documentation Index
> Fetch the complete documentation index at: https://docs.xpander.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# AI Gateway

> One address for every model call your organization makes, from agents, apps and coding tools, with your keys, models, budgets and records applied.

Every model call your organization makes goes through the AI Gateway. Agents, apps and the coding tools people run (Claude Code, Codex and OpenCode) call it with an xpander key or session token, never a provider key.

![Diagram](https://docs.xpander.ai/images/diagrams/run-llm-gateway-index-5a49e54f.svg)

## What happens to each call

| Step | What the gateway does |
| - | - |
| 1. **Who is calling** | Identifies the caller from the key: a person's API key, an agent's run or an organization key. Unknown, revoked or expired keys are refused |
| 2. **What they may use** | Checks the model (or [alias](https://docs.xpander.ai/run/llm-gateway/providers-and-keys.md#aliases-and-fallbacks)) against the models on for the person's groups |
| 3. **Who pays** | The person's own Claude plan (where enabled), your stored key, or xpander's key in credits ([who pays](https://docs.xpander.ai/run/llm-gateway/which-key-pays.md)) |
| 4. **Budgets** | Checks organization, group, person, agent and key [budgets](https://docs.xpander.ai/run/llm-gateway/budgets-and-blocks.md) before the call leaves |
| 5. **Fallbacks** | Retries an overloaded or failing provider, then tries the alias's next model |
| 6. **The record** | One [record](https://docs.xpander.ai/run/llm-gateway/requests-and-costs.md) per call: who, its **Source** (what started it, then the coding tool or harness), model, provider, tokens, cost, timings |

Prompt and response text is saved encrypted, unless an admin turns off [Saved prompt text](https://docs.xpander.ai/run/llm-gateway/policies.md#prompt-and-response-text). Calls xpander makes for you, such as conversation summaries, show in Logs with a **System** badge.

## Agent runs

An agent session holds no provider key. Its harness gets `ANTHROPIC_BASE_URL` (Claude Code) or `OPENAI_BASE_URL` (Codex) pointing at a model relay on its own executor, and only a placeholder for the run's session token. The relay puts the token on each model call and forwards it to the LLM proxy on the Agent Controller, so the commands and skills the agent runs never see a credential.

> [accordion] What the proxy does and what passes through it
>
> ![Diagram](https://docs.xpander.ai/images/diagrams/run-llm-gateway-index-ebf4bdcb.svg)
>
> No executor, workspace or harness holds a provider key. For each call, the proxy resolves the credential, applies the model policy and budget, records the call under the person who asked, and forwards it (through the in-cluster AI Gateway on a self-hosted install). It carries:
>
> * Every turn of an agent on Claude Code, Codex or OpenCode.
> * xpander's own model calls: thread titles, context compaction, evaluation judges, and the **gateway model** that filters entries ahead of the agent's model.
> * Guardrail moderation, when enabled.

## Any model on any harness

The gateway translates between harness and provider APIs, so Claude Code, Codex and OpenCode can each run any model your organization allows, including your own (such as vLLM on your GPUs on Air-Gapped). Pick it in the composer's [**Harness and model**](https://docs.xpander.ai/use/agents/create.md#harness-and-model) picker.

> [accordion] Which models a harness lists
>
> * Claude Code and Codex list their own vendor's models, plus a model from another vendor that passed xpander's test on that harness: chat, a tool call, streaming and long context. OpenCode lists every catalog model except a pair the test showed as broken.
> * A model from your own provider is listed for a harness after an admin runs **Test model** on it for that harness and it passes ([Model testing](https://docs.xpander.ai/run/llm-gateway/model-testing.md)).
> * Each turn is recorded in **Platform > AI governance > Logs** with the model that ran it.
> * A personal Claude plan applies only to Claude models from Anthropic. A model from another vendor runs on the organization's providers.

## Cloud, Hybrid and Air-Gapped

On xpander cloud the gateway runs in xpander cloud at `https://llm-gw.xpander.ai`. On Hybrid it runs in your install for your agents, while people's coding tools use xpander cloud. On Air-Gapped everything runs in your install, at `https://llm-gw.<your domain>`.

> [accordion] Address, settings, who pays and records, per deployment
>
> | | xpander cloud | Hybrid | Air-Gapped |
> | - | - | - | - |
> | Where the gateway runs | xpander cloud | Your install, for your agents. People's coding tools use xpander cloud | Your install |
> | Address for coding tools and SDKs | `https://llm-gw.xpander.ai` | `https://llm-gw.xpander.ai` | `https://llm-gw.<your domain>` |
> | Settings | [Platform](https://platform.xpander.ai/governance/model-access) | Platform, reaching your install within seconds | Your install's Platform |
> | Who pays | Own Claude plan (when enabled), your keys, or xpander's keys (credits) | Own Claude plan (when enabled) and your keys. Never xpander's keys | Your keys and your install's credentials |
> | Request records | Your organization's database in xpander cloud | Agents: your install. Coding tools: xpander cloud | Your install |

On Hybrid and Air-Gapped the gateway is always on, and every provider your agents use needs a key. See [Self-hosted](https://docs.xpander.ai/run/llm-gateway/self-hosted.md).

## Where each setting lives

Admins work under **Platform > AI governance**: **Models**, **Budgets**, **Policies**, **Usage**, **Logs** and **Audit log**.

> [accordion] What each page holds
>
> | Under **Platform > AI governance** | Holds |
> | - | - |
> | [Models](https://platform.xpander.ai/governance/model-access) | Providers, keys, Azure endpoints, Bedrock roles, model access and aliases ([Providers and keys](https://docs.xpander.ai/run/llm-gateway/providers-and-keys.md)) |
> | [Budgets](https://platform.xpander.ai/governance/budgets) | Spending limits ([Budgets](https://docs.xpander.ai/run/llm-gateway/budgets-and-blocks.md)) |
> | [Policies](https://platform.xpander.ai/governance/policies) | License models, key lifetime, request and conversation retention, saved prompt text ([Policies](https://docs.xpander.ai/run/llm-gateway/policies.md)) |
> | [Usage](https://platform.xpander.ai/governance/usage) and [Logs](https://platform.xpander.ai/governance/logs) | Spend and requests ([Usage and logs](https://docs.xpander.ai/run/llm-gateway/requests-and-costs.md), [Log destinations](https://docs.xpander.ai/run/llm-gateway/logs-export.md)) |
> | [Audit log](https://platform.xpander.ai/governance/audit) | Who changed what |

People's own keys, coding tool setup and spend are in xpander Chat: [AI models & keys](https://docs.xpander.ai/use/ai-models-and-keys.md) and [Usage](https://chat.xpander.ai/?settings=my-usage). On a self-hosted install, use your own addresses.

## Get started

1. In [Models](https://platform.xpander.ai/governance/model-access), choose **Add provider** and add a key (on xpander cloud you can start on the models included with xpander), then tick the models people may use. **Who may use it…** limits one to groups. Optionally add an alias, such as `default-coding` with a fallback.
2. Set [Budgets](https://platform.xpander.ai/governance/budgets) and [Policies](https://platform.xpander.ai/governance/policies).
3. In [AI models & keys](https://chat.xpander.ai/?settings=my-gateway), choose a client and **Copy setup command** ([steps](https://docs.xpander.ai/use/ai-models-and-keys.md#connect-a-coding-tool)). Your first request shows at the top of [Logs](https://platform.xpander.ai/governance/logs).

## Next steps

> [card] [Fix a refused call](https://docs.xpander.ai/run/llm-gateway/errors.md)
>
> Messages and fixes.

> [card] [See what is stored](https://docs.xpander.ai/run/llm-gateway/security.md)
>
> Security of gateway records.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.