> ## Documentation Index
> Fetch the complete documentation index at: https://docs.xpander.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# AI gateway

> Every model call from every agent goes through xpander's LLM proxy on the Agent Controller: what it enforces, where it is configured, and how to route it through a gateway you already run.

The AI gateway is the LLM proxy on the Agent Controller, the xpander service that mediates every model and skill call. No executor, workspace or harness holds a provider key. The client inside the session is pointed at the proxy instead. The controller then resolves the credential, applies the organization's model policy and budget, records the call under the person who asked, and forwards it to the provider.

## What passes through it

* Every turn of an agent on Claude Code, Codex or OpenCode. The session receives `ANTHROPIC_BASE_URL` (Claude Code) or `OPENAI_BASE_URL` (Codex) pointing at the proxy, with a session token instead of a provider key.
* xpander's own model calls: thread titles, context compaction, evaluation judges, and the **gateway model** that runs entry filtering ahead of the agent's model.
* Guardrail moderation, when enabled.

Preflight refuses a self-hosted install that has no provider configured for model calls: an organization default provider, a gateway provider, or a per-provider in-cluster endpoint must exist.

## Where it is configured

| What                                                                             | Where                                                                                                                                                                   |
| -------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Providers, your own keys, the organization default model, model access per group | **Settings > LLM settings**                                                                                                                                             |
| Spending limit, enforcement, per-agent ceiling, alerts                           | **Settings > Usage** (**Settings > Budget** on subscription-billed organizations); see [Budgets](/use/agents/budgets)                                                   |
| Usage per agent and per person                                                   | **Settings > Usage**                                                                                                                                                    |
| Install-level defaults on a self-hosted cluster                                  | `global.llm.defaults.{provider,model}` and `global.llm.gateway.{provider,model,apiBase}` in the chart values                                                            |
| Amazon Bedrock                                                                   | `global.llm.bedrock.region`, then `bearerTokenSecret`, `assumeRoleArn` (+ `assumeRoleExternalId`), or nothing for EKS Pod Identity / IRSA; `baseUrl` for a VPC endpoint |

Which harness a new agent starts on follows the organization's default provider: Anthropic maps to Claude Code, Amazon Bedrock with an Anthropic model to Claude Code on Bedrock, OpenAI to Codex, any other provider to OpenCode on that model.

<Frame caption="Settings > LLM settings: the providers card, workspace defaults and gateway model. Shown with sample data.">
  <img src="https://mintcdn.com/xpanderai-099931d1/OdLxX9OyFVicAV0l/images/verify/llm-settings-1.png?fit=max&auto=format&n=OdLxX9OyFVicAV0l&q=85&s=a80938393c52bcfae3d72a39c4d0da56" alt="LLM settings providers card" width="1440" height="900" data-path="images/verify/llm-settings-1.png" />
</Frame>

## Routing through a gateway you already run

The proxy's upstream per provider is an environment variable on the Agent Controller, so an existing AI gateway in your network can sit between xpander and the vendor:

| Provider                                                               | Controller environment                                                                    |
| ---------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| Anthropic                                                              | `LLM_PROXY_ANTHROPIC_BASE_URL`                                                            |
| Amazon Bedrock                                                         | `LLM_PROXY_AMAZON_BEDROCK_BASE_URL` (a VPC or PrivateLink runtime endpoint on Air-Gapped) |
| Any OpenAI-compatible runtime (vLLM, Ollama, NVIDIA NIM, your gateway) | The provider's `apiBase` under `global.llm`                                               |

The controller egresses to `api.anthropic.com` by default; with an override set, that is the only host it needs for Anthropic. xpander's model policy, budget and record are applied on the controller before the call leaves; the gateway sees one more client, and its other clients stay outside xpander's controls.

## Verify

```bash theme={"dark"}
kubectl -n xpander port-forward service/xpander-agent-controller 9016:9016 &
curl http://localhost:9016/health
```

In Xpander Chat, **Settings > Locations** shows the installation as **Connected**. A model call from any agent appears in the task's usage and under **Settings > Usage**.

Deep pages: [AI vendors](/run/integrations/ai-vendors#provider-setup-on-a-self-hosted-install), [Air-gapped: local LLM inference](/self-hosted/airgap#local-llm-inference), [Hybrid: managing LLM API keys](/self-hosted/hybrid#managing-llm-api-keys).