> ## Documentation Index
> Fetch the complete documentation index at: https://docs.xpander.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Providers, keys and model access

> Connect model providers and your own endpoints, store keys, turn models on, decide who may use them, add aliases with fallbacks, and choose whose key pays.

Set up providers, keys, model access and aliases in [Models](https://platform.xpander.ai/governance/model-access) (**Platform > AI governance > Models**). Keys stay in your organization's vault, and the AI Gateway adds them to each call on its way out, so no person, agent or desktop receives one.

![Diagram](https://docs.xpander.ai/images/diagrams/run-llm-gateway-providers-and-keys-b544bf2d.svg)

## The Models page

![Models page with the gateway base URL and the Providers list: each provider's details, status and how many models are on](https://mintcdn.com/xpanderai-099931d1/H-WYYv839kjeMNSx/images/llm-gateway/05-models-page.png?fit=max\&auto=format\&n=H-WYYv839kjeMNSx\&q=85\&s=d10190fad6f88cbe1f4c95c237f54613)

* **Base URL**: the address clients and SDKs call. On a self-hosted install with no published host, the in-cluster address: `http://llm-gw:4000` on Air-Gapped, `http://xpander-llm-gw:4000` on Hybrid with the release name `xpander` ([release names](https://docs.xpander.ai/self-hosted/hybrid.md#what-gets-deployed)).
* **Get started**: a checklist, hidden once the required steps are done.
* **Providers**: each with **Connected**, **Needs a key**, **All models off** or **Key error**.
* **Organization defaults**: what new agents start with ([below](#defaults-for-new-agents)).
* **Who can use what**: who can call which model ([below](#who-may-use-which-models)).

## Add a provider

| Install | **Add provider** offers | Add a key to a listed provider |
| - | - | - |
| xpander cloud | **Model vendor** and **Custom endpoint** | **Use your own key…** in the row's menu, or **Add key** in its card |
| Hybrid or Air-Gapped | A custom endpoint | **Use your own key…** in the row's menu |

On xpander cloud, choose **Add provider**, pick the **Provider** (such as Anthropic, OpenAI, Google Gemini, Amazon Bedrock, Microsoft Foundry, OpenRouter, Cerebras or Z.ai for GLM models), paste the **API key**, name it in uppercase and underscores (such as `PROD_ANTHROPIC`), and choose **Add provider**. Your own keys need the Enterprise plan; without it the page shows *Bring-your-own LLM keys is an Enterprise feature* with **Contact us**.

![Add Anthropic key panel with API key, Name and Description fields and an Add Anthropic key button](https://mintcdn.com/xpanderai-099931d1/wqIIXSsh0kqA2hZ_/images/verify/llm-add-key-anthropic-1.png?fit=max\&auto=format\&n=wqIIXSsh0kqA2hZ_\&q=85\&s=aab93b66f04150b4602a5d0a77506d7e)

Use your own key on a built-in provider: the API key, its name and an optional description. Shown with sample data.

> [accordion] Providers and their install key names
>
> | Provider | Key | Notes |
> | - | - | - |
> | Anthropic | `ANTHROPIC_API_KEY` | Default for Claude Code. Baked default model: Fable 5.1 (`claude-fable-5-1`) |
> | OpenAI | `AGENTS_OPENAI_API_KEY` | Default for Codex. Baked default: `gpt-6-astra` |
> | Amazon Bedrock | `AWS_BEARER_TOKEN_BEDROCK`, or keyless through the pods' AWS identity | Fable- and Mythos-class Anthropic models need the data-retention opt-in ([Amazon Bedrock](#amazon-bedrock)) |
> | Google | `GOOGLE_API_KEY` | |
> | Fireworks, Nebius, OpenRouter, Helicone | `FIREWORKS_API_KEY`, `NEBIUS_API_KEY`, `OPENROUTER_API_KEY`, `HELICONE_API_KEY` | OpenCode takes any `<provider>/<model>` slot the proxy routes |
> | Cerebras | `CEREBRAS_API_KEY` | |
> | Z.ai | `Z_AI_API_KEY` | GLM models. Hybrid chart 0.23.3 has no `secrets.static` value for it, so set it with `envFromSecretKeys` |
> | Your own OpenAI-compatible runtime (vLLM, Ollama, NVIDIA NIM) | None, or whatever the runtime requires | Required on Air-Gapped. Add it as a [custom endpoint](#custom-endpoints) |

### Amazon Bedrock

Set **Credential** to **IAM role** to store no secret: xpander assumes the role for short-lived credentials on each use, with your organization ID as the external ID on xpander cloud. Enter the **Role ARN** and **Region**, choose **Test connection**, then **Add role**. Full steps: [Amazon Bedrock with an IAM role](https://docs.xpander.ai/run/integrations/bedrock-iam-role.md). Fable- and Mythos-class models also need a per-region data-retention opt-in.

> [accordion] Data-retention opt-in for Fable- and Mythos-class models
>
> **Fable- and Mythos-class Anthropic models on Bedrock need the account's data-retention opt-in, per region.**
>
> * With the default mode, every call fails with `400 data retention mode 'default' is not available for this model`. Accepted modes: `provider_data_share`, or `aws_review` where offered.
> * There is no console switch. Per region, call `PUT https://bedrock.<region>.amazonaws.com/data-retention` with `{"mode": "provider_data_share"}` (SigV4-signed, IAM action `bedrock:PutAccountDataRetention`), for every region a `global.` inference profile can route to.
> * The first call after the change can still fail while it propagates. Retry first.
> * It applies to **whichever AWS account signs the call**. An organization's own stored Bedrock key wins over the install's credential, so that account needs the opt-in too.
> * Older Anthropic models need no opt-in, which is why the error looks model-specific.

> [accordion] Bedrock on a self-hosted install: auth methods and settings
>
> On Hybrid, Bedrock needs your own credentials: a Bedrock key or role under **Platform > AI governance > Models**, or the cluster's own AWS identity, and the account needs the opt-in above.
>
> Claude Code runs Anthropic models on Bedrock through the proxy's `amazon_bedrock` leg. The Agent Controller resolves AWS auth in this order, and no executor sees it:
>
> | Method | Controller environment | Notes |
> | - | - | - |
> | Bearer token | `AWS_BEARER_TOKEN_BEDROCK` (+ `BEDROCK_REGION` / `AWS_REGION`) | Relayed as-is, no signing. The simplest option |
> | Assume role | `BEDROCK_ASSUME_ROLE_ARN` (+ `BEDROCK_ASSUME_ROLE_EXTERNAL_ID`, `_SESSION_NAME`) | STS credentials, self-refreshing, SigV4 signed |
> | Static keys | `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` (/ `AWS_SESSION_TOKEN`) | SigV4 signed |
> | Default credential chain | `AGENT_GATEWAY_BEDROCK_DEFAULT_CHAIN=true` (default) | IRSA / EKS Pod Identity / instance profile, SigV4 signed |
>
> | Install | Setting |
> | - | - |
> | Air-Gapped | No public Bedrock. Point `LLM_PROXY_AMAZON_BEDROCK_BASE_URL` at a VPC (PrivateLink) Bedrock runtime endpoint with the matching region, or leave Bedrock off. The chart exposes this under `global.llm.bedrock`: `region`, `bearerTokenSecret` (a Secret you create with a Bedrock API key), `assumeRoleArn` and `assumeRoleExternalId` (STS), and `baseUrl` (the VPC endpoint). Leave the token and role empty to use the pod's credential chain (EKS Pod Identity or IRSA). |
> | Hybrid | Set the same variables on the Agent Controller through the chart's env values. |
>
> The keyless methods apply to the proxy on the Agent Controller. The `agent-worker` service has its own constraint on keyless Bedrock: see [Amazon Bedrock: key vs IAM](https://docs.xpander.ai/self-hosted/hybrid-llm-keys.md#amazon-bedrock-keyless-iam-or-api-key).

### Azure endpoint

Each **Microsoft Foundry** key needs its resource's **Azure endpoint** (from **Keys and Endpoint** in the Azure portal, such as `https://my-resource.openai.azure.com`); **API version** (such as `2024-10-21`) is optional. Use **Set endpoint** or **Edit endpoint** in the key's menu. A key without one shows **No endpoint** and its calls are refused. On a self-hosted install with an Azure endpoint set at install time, a key's own endpoint is optional.

### Custom endpoints

Connect a model server or gateway you run (vLLM, Ollama, NVIDIA NIM, an internal gateway). Any admin who can add provider keys can. Choose **Add provider** (then **Custom endpoint** on xpander cloud).

![Add provider form with Name Internal vLLM, ID internal\_vllm, Protocol OpenAI-compatible, a Base URL ending in /v1, API key No key, and Models with Test connection and Add model](https://mintcdn.com/xpanderai-099931d1/H-WYYv839kjeMNSx/images/llm-gateway/06-custom-endpoint-form.png?fit=max\&auto=format\&n=H-WYYv839kjeMNSx\&q=85\&s=291624eb4f0d1060402a67822b7cc177)

Adding a custom endpoint: name, ID, protocol, base URL, API key and models.

**1. Name it**

Enter a **Name**. The **ID (used in API calls)** is derived from it and can't change once added.

**2. Pick the protocol and base URL**

**OpenAI-compatible** or **Anthropic-compatible**, and the **Base URL** with its `/v1` path, as the pods reach it: public `https` on xpander cloud (`https://ai-gateway.example.com/v1`), inside your network on Hybrid or Air-Gapped (`http://vllm.ai-tools.svc.cluster.local:8000/v1`). See [address rules](#model-endpoints-on-xpander-cloud).

**3. Pick a key**

**No key (local or unauthenticated server)**, **Use an existing key** or **Create a new key**.

**4. List the models and save**

**Test connection** reports **Reachable** and lists the models (a `tools` badge marks function calling), or **Add model** adds IDs. Choose **Add provider**. The models are on for your organization at once.

For a gateway your company already runs, see [Your existing AI gateway](https://docs.xpander.ai/run/llm-gateway/existing-gateway.md).

To check that Claude Code, Codex and OpenCode work with a model, choose **Test model** in its row's menu. See [Model testing](https://docs.xpander.ai/run/llm-gateway/model-testing.md).

> [accordion] Prices, menu actions and newly served models
>
> To track spend, set a price per model with **Edit model** under **Keys and models…** (**Price per 1M tokens (USD, optional)**). Prices track spend only and are never charged. A model with no price reads as zero spend, so leave it at 0 if you don't pay per token. The row's menu also has **Check now**, **Edit provider** and **Remove provider**. New models that a vLLM, SGLang or NVIDIA NIM server starts serving are found with their context length and wait for an admin to approve them. Models already in use stay approved.

> [accordion] Endpoint health checks
>
> Every 2 minutes, the gateway runs the same agent check on one of each custom endpoint's models (xpander's own endpoints are not checked).
>
> | When | Result |
> | - | - |
> | Two failed checks in a row | Marked **down** |
> | 5 or more provider errors in one 2-minute window, with no more successes than errors | Marked **down** by agent traffic. Refused keys, bad requests and full contexts don't count. |
> | One passing check | Healthy again |
>
> A down endpoint shows a red banner in [Models](https://platform.xpander.ai/governance/model-access) with the failed step, the cause, the fix and the last 24 checks. **Open in Test model** reruns everything. Model admins are notified in the app, by email and on Slack when it goes down and when it recovers.

> [accordion] How the gateway adapts requests to each model
>
> Each custom model has a **profile** that reshapes every request to fit it. Open it with **View profile** (**Edit profile** if you may change it).
>
> | Setting | Options |
> | - | - |
> | **Thinking** | Pass, Strip, Map to reasoning effort |
> | **Effort** | Pass, Strip, Map to `reasoning_effort`, Map to `chat_template_kwargs` (the `enable_thinking` and `thinking` switches) |
> | **System prompt** | Pass, Merge into the first message |
> | `Max tools` | Refuse requests with more function definitions, before the model runs |
> | **Context length** | Refuse requests that don't fit, before the model runs |
>
> A context-full refusal uses each CLI's own error, so the CLI compacts as usual: OpenAI's `context_length_exceeded` for Codex and OpenCode, Anthropic's "prompt is too long" for Claude Code.
>
> Values come from four sources: **Family default** (built in for a model family), **Learned by Test model** (a check that found a fix saves it), **Set by an admin** (wins over the other two), and **Discovered** (the context length the endpoint reports, used when nothing else sets one). Family defaults: gpt-oss maps effort to `reasoning_effort`. Qwen, DeepSeek and GLM map it to `chat_template_kwargs`. Kimi and Nemotron strip thinking and effort, and Kimi accepts at most 128 function definitions. Each model's context length comes from the endpoint's model list (`/v1/models`, the `max_model_len` field on vLLM). The engine version comes from vLLM (`/version`), SGLang (`/get_server_info`) or NVIDIA NIM (`/v1/version`).

> [accordion] A server with no key
>
> With **No key (local or unauthenticated server)**, every call carries a placeholder, never a real key. Gateway callers name the model `<ID>::<model>`, such as `internal_vllm::llama-3.3-70b`. **Check now** and **Test connection** still work. Calls show in [Logs](https://platform.xpander.ai/governance/logs) at the price set with **Edit model**, and are not charged in credits.
>
> > [!WARNING]
> >
> > On xpander cloud the endpoint must be public, so anyone who finds its address can use it. Protect the server with a key, or connect it from a Hybrid or Air-Gapped install, where it can stay private.

> [accordion] Using a custom provider in agents
>
> * `POST /v1/agents` accepts only built-in providers in `model_provider`. A custom ID gets `422`.
> * Pick a custom provider per conversation in the composer's **Harness and model** popover (**Claude Code**, **Codex**, **OpenCode**).
> * A local model needs a context window well above the prompt xpander composes. Worked example: [Air-Gapped: a worked vLLM setup](https://docs.xpander.ai/self-hosted/airgap-models.md#a-worked-vllm-setup-on-eks-auto-mode).
> * With no `/responses` route on the endpoint, the gateway sends Codex calls as chat completions.

### Model endpoints on xpander cloud

On xpander cloud, a model endpoint must be a public `https` address. On Hybrid or Air-Gapped it can be private, but never link-local or cloud metadata. Both are checked when saved and on every call.

> [accordion] Allowed and refused addresses
>
> | Install | Allowed | Refused, when saved and again on every call |
> | - | - | - |
> | xpander cloud | Public `https` addresses only | Private, loopback, link-local, cloud metadata and cluster addresses: *Model endpoints are called at public addresses here, so this endpoint was not saved. Use the endpoint's public https address, or keep the provider's own endpoint.* |
> | Hybrid or Air-Gapped | Private addresses inside your network | Link-local and cloud metadata addresses (`169.254.0.0/16`, `fe80::/10`, the cloud metadata hosts), checked against the address actually dialled |
>
> This applies to Azure endpoints, OpenAI-compatible endpoints and model endpoints set on an agent.

## Turn models on

In a provider's row (or **Keys and models…**), tick the models the organization may use. Prices are USD per 1M tokens. **Disable provider** or **Disable model** stops it everywhere. A model blocked at the gateway shows **Off** (*No one in the organization can use this model.*) until **Unblock at the gateway**.

## Who may use which models

![Access panel for Amazon Bedrock listing Organization and the groups Default, Design, Engineering and Sales, each with All models ticked](https://mintcdn.com/xpanderai-099931d1/H-WYYv839kjeMNSx/images/llm-gateway/07-who-may-use-it.png?fit=max\&auto=format\&n=H-WYYv839kjeMNSx\&q=85\&s=70ba6a0afe2e842e00fd684757fd69ff)

Who may use it: the organization and each group, with which of the provider's models they may use. Shown with sample data.

**Who may use it…** in a provider's or model's menu sets what the **Organization** and each group may use: **All models** (every model the provider has on) or a list, within what the organization has on. A model outside a group's list is hidden from its members' pickers, and a run naming it is refused.

**Who can use what** answers "why can't this person use this model" **By person**, **By group** or **By model**, with the gateway's own reason: **Blocked for everyone**, **Group switched off**, **Not on the organization's list** or **Not on the group's list**.

## Aliases and fallbacks

An alias is a model name you define, such as `default-coding`. Clients send it as the model and the gateway calls its target, so changing the target moves every client at once.

![Diagram](https://docs.xpander.ai/images/diagrams/run-llm-gateway-providers-and-keys-edd3d8e7.svg)

Add one with **Add alias…** in a model's **⋯** menu. An overload or server error is retried on the same model, then on each fallback in order; a failure that will not change (invalid request, model off, spent budget) is not. Each fallback uses the key assigned for its own provider, and each attempt shows under **Routing** in [Logs](https://platform.xpander.ai/governance/logs). Callers are still limited to the models they may use.

> [accordion] Add an alias: names, target and fallbacks
>
> **1. Start from a model**
>
> Open the provider, then the model's **⋯** menu, and choose **Add alias…**.
>
> **2. Name it and choose the target**
>
> Enter a **Name**: lowercase letters, digits, dots, dashes and underscores, starting with a letter or digit, 63 characters at most. A **Description** is optional. **Points to** starts on the model you chose. Pick any other model you turned on.
>
> **3. Add fallbacks and save**
>
> Under **Fallbacks, tried in order**, choose **Add a fallback model** for each model to try. The target can't be a fallback, and each fallback appears once. Choose **Add alias**.
>
> Names starting with `xp-` are reserved: *Names starting with xp- are xpander's built-in models; choose another name for: ...*

> [accordion] Read, edit and remove aliases, and where people find them
>
> Each alias is a tag under its model. Aliases whose model is not listed join **Other aliases**. A grayed-out tag means the target is off. A warning dot means a fallback is off. Hover for details. Click a tag to edit it and choose **Save alias**, or use **Delete alias** in the card's menu. Clients still sending a deleted alias get an error.
>
> People find aliases under **Shortcuts your admin set up** in [AI models & keys](https://chat.xpander.ai/?settings=my-gateway), in the **Model** picker of Codex, OpenCode and SDK setups, and in `/v1/models` on the gateway address.

## Whose key pays

A provider can hold several keys, such as one per team. In its card under **AI Gateway**, set the **Organization default** (or **Automatic**), assign keys in **Give a group, person or agent its own key**, and confirm with **Check a person or agent**. Each call uses the first key assigned to:

1. the agent making the call
2. the person making the call
3. the person's primary group
4. the organization default for that provider

Whether a stored key or xpander's key (billed in credits) pays at all depends on your [license model](https://docs.xpander.ai/run/llm-gateway/which-key-pays.md). To rotate, assign a new key or replace the value in the vault; the next call uses it.

> [accordion] Key labels and rotation details
>
> * **Rejected by the provider** marks a key the provider refused. **Used by the gateway** marks the one in use.
> * Calls on keys xpander provides are charged in credits. Calls on a key you brought, agent or organization level, never are.
> * A provider with no key of your own shows the **xpander key** row (*Metered in credits.*) on xpander cloud, **Installation credentials** on a self-hosted install with a key in `secrets.static`, and **Needs a key** otherwise. Calls to a provider that needs a key are refused with a message that names it.
> * A deleted key stops being used from the next call.

### Keys on a self-hosted install

The gateway is always on self-hosted, so every provider your agents use needs a key, the organization's or the install's, or calls are refused naming it. Hybrid agents never use xpander's keys.

> [accordion] Where an agent's key comes from, and how to set an install key
>
> ![Diagram](https://docs.xpander.ai/images/diagrams/run-llm-gateway-providers-and-keys-d643422c.svg)
>
> Every agent model call, including Bedrock and per-agent endpoints, goes through the gateway. The key comes from the first of: a credential set on one agent, the organization's key for the provider (above), or the install's keys in chart values `secrets.static.<provider>ApiKey` or a Kubernetes secret referenced with `envFromSecretKeys`. The keys a call can use are shown in the **Providers** table as **Connected** or **Needs a key**.
>
> ```bash theme={"dark"}
> helm upgrade xpander xpander/xpander --namespace xpander --reset-then-reuse-values \
>   --set secrets.static.anthropicApiKey=<KEY> --timeout 5m
> ```
>
> * `--reset-then-reuse-values` needs Helm 3.14 or later. It keeps your values and applies the chart's defaults for the rest.
> * Provider credentials live **only on the Agent Controller**. An Anthropic key there egresses to `api.anthropic.com`; to use your own gateway, set `LLM_PROXY_ANTHROPIC_BASE_URL` ([Route the controller through your gateway](https://docs.xpander.ai/run/llm-gateway/existing-gateway.md#route-a-self-hosted-install-through-your-gateway)).
> * A key named by an environment variable on agent-controller (custom provider, agent or organization default model) never reads the install's own credentials. A name on the reserved list reads as no key ([reserved key names](https://docs.xpander.ai/self-hosted/hybrid-llm-keys.md#key-names-read-from-an-environment-variable)).
> * Pods restart on their own when a secret changes. The `xpander-static` secret has a Helm keep policy: if a key is not picked up after an upgrade, patch the secret and restart `xpander-agent-worker`. Full mapping: [Hybrid: managing LLM API keys](https://docs.xpander.ai/self-hosted/hybrid-llm-keys.md#managing-llm-api-keys).
> * Install-level defaults: `global.llm.defaults.{provider,model}` and `global.llm.gateway.{provider,model,apiBase}` in the chart values. Bedrock: `global.llm.bedrock` ([Amazon Bedrock](#amazon-bedrock)).

## Defaults for new agents

The organization's default provider picks a new agent's harness, unless the create names one. Changing defaults does not touch existing agents.

> [accordion] Which harness and model a new agent gets, and the install knobs
>
> A create that names a runtime, provider, model or harness settings is honored as written. Otherwise it is **defaulted**, the same way on cloud, Hybrid and Air-Gapped:
>
> 1. **The organization's default LLM** picks the CLI:
>
>    | Organization default LLM | CLI |
>    | - | - |
>    | Anthropic | Claude Code, on that model |
>    | Amazon Bedrock with an Anthropic model id | Claude Code on Bedrock |
>    | OpenAI | Codex |
>    | Any other provider the LLM proxy routes | OpenCode, with that `<provider>/<model>` slot |
>
>    No default, or a provider no CLI runs, falls through.
> 2. **The install's default knobs** (env on the `agents` and `agent-controller` services, the chart values below on Air-Gapped).
> 3. **The baked defaults**: Claude Code on Anthropic direct with Fable 5.1 (`claude-fable-5-1`), Codex on `gpt-6-astra`, OpenCode on `anthropic/claude-opus-5`.
>
> So a fresh Air-Gapped install with no default LLM creates Claude Code agents on **Fable 5.1 over Anthropic direct**, and needs an Anthropic key on the controller or Bedrock set as the default (with the opt-in above).
>
> | Deployment | Knob | Effect |
> | - | - | - |
> | Air-Gapped | `global.harness.defaultModels.{claudeCode,codex,opencode,xpander}` (rendered as `XPANDER_HARNESS_DEFAULT_MODEL_<CLI>` on `agent-controller` and the `agents` Deployment) | The model a defaulted agent gets when the org has no default LLM. Name models your endpoint serves. `claudeCode` and `codex` take a bare model id. `opencode` and `xpander` take a `<provider>/<model>` slot. A Bedrock inference-profile id in `claudeCode` (for example `us.anthropic.claude-sonnet-4-6`) implies the provider. An explicit `agents.env` / `agent-controller.env` entry wins |
> | Air-Gapped | `global.harness.defaultProviders.claudeCode` (rendered as `XPANDER_HARNESS_DEFAULT_PROVIDER_CLAUDE_CODE` on the `agents` Deployment) | Empty = Anthropic direct. `amazon_bedrock` sends defaulted Claude Code agents to Bedrock through `global.llm.bedrock` |
> | Hybrid | none in the chart | The `agents` service runs in xpander cloud, so defaulted Claude Code agents run on **Anthropic direct**. Set the org default LLM to change that. The controller needs credentials for whichever provider your agents use |
>
> **Claude Code on Bedrock on Air-Gapped needs both harness defaults:** `global.harness.defaultProviders.claudeCode: amazon_bedrock` and `global.harness.defaultModels.claudeCode: <Bedrock model id>`. Without them, the first turn fails with `503 no anthropic credential configured for harness agents`, even when the organization's default model is on Bedrock.

## Next steps

> [card] [Decide who pays for each call](https://docs.xpander.ai/run/llm-gateway/which-key-pays.md)
>
> License models and stored keys.

> [card] [Connect Bedrock with an IAM role](https://docs.xpander.ai/run/integrations/bedrock-iam-role.md)
>
> No long-lived keys.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.