Skip to main content
Set up providers, keys, model access and aliases in Models (Platform > AI governance > Models). Keys stay in your organization’s vault, and the AI Gateway adds them to each call on its way out, so no person, agent or desktop receives one.

​
The Models page

Models page with the gateway base URL and the Providers list: each provider's details, status and how many models are on
  • Base URL: the address clients and SDKs call. On a self-hosted install with no published host, the in-cluster address: http://llm-gw:4000 on Air-Gapped, http://xpander-llm-gw:4000 on Hybrid with the release name xpander (release names).
  • Get started: a checklist, hidden once the required steps are done.
  • Providers: each with Connected, Needs a key, All models off or Key error.
  • Organization defaults: what new agents start with (below).
  • Who can use what: who can call which model (below).

​
Add a provider

On xpander cloud, choose Add provider, pick the Provider (such as Anthropic, OpenAI, Google Gemini, Amazon Bedrock, Microsoft Foundry, OpenRouter, Cerebras or Z.ai for GLM models), paste the API key, name it in uppercase and underscores (such as PROD_ANTHROPIC), and choose Add provider. Your own keys need the Enterprise plan; without it the page shows Bring-your-own LLM keys is an Enterprise feature with Contact us.
Add Anthropic key panel with API key, Name and Description fields and an Add Anthropic key button

Use your own key on a built-in provider: the API key, its name and an optional description. Shown with sample data.

​
Amazon Bedrock

Set Credential to IAM role to store no secret: xpander assumes the role for short-lived credentials on each use, with your organization ID as the external ID on xpander cloud. Enter the Role ARN and Region, choose Test connection, then Add role. Full steps: Amazon Bedrock with an IAM role. Fable- and Mythos-class models also need a per-region data-retention opt-in.
Fable- and Mythos-class Anthropic models on Bedrock need the account’s data-retention opt-in, per region.
  • With the default mode, every call fails with 400 data retention mode 'default' is not available for this model. Accepted modes: provider_data_share, or aws_review where offered.
  • There is no console switch. Per region, call PUT https://bedrock.<region>.amazonaws.com/data-retention with {"mode": "provider_data_share"} (SigV4-signed, IAM action bedrock:PutAccountDataRetention), for every region a global. inference profile can route to.
  • The first call after the change can still fail while it propagates. Retry first.
  • It applies to whichever AWS account signs the call. An organization’s own stored Bedrock key wins over the install’s credential, so that account needs the opt-in too.
  • Older Anthropic models need no opt-in, which is why the error looks model-specific.
On Hybrid, Bedrock needs your own credentials: a Bedrock key or role under Platform > AI governance > Models, or the cluster’s own AWS identity, and the account needs the opt-in above.Claude Code runs Anthropic models on Bedrock through the proxy’s amazon_bedrock leg. The Agent Controller resolves AWS auth in this order, and no executor sees it:The keyless methods apply to the proxy on the Agent Controller. The agent-worker service has its own constraint on keyless Bedrock: see Amazon Bedrock: key vs IAM.

​
Azure endpoint

Each Microsoft Foundry key needs its resource’s Azure endpoint (from Keys and Endpoint in the Azure portal, such as https://my-resource.openai.azure.com); API version (such as 2024-10-21) is optional. Use Set endpoint or Edit endpoint in the key’s menu. A key without one shows No endpoint and its calls are refused. On a self-hosted install with an Azure endpoint set at install time, a key’s own endpoint is optional.

​
Custom endpoints

Connect a model server or gateway you run (vLLM, Ollama, NVIDIA NIM, an internal gateway). Any admin who can add provider keys can. Choose Add provider (then Custom endpoint on xpander cloud).
Add provider form with Name Internal vLLM, ID internal_vllm, Protocol OpenAI-compatible, a Base URL ending in /v1, API key No key, and Models with Test connection and Add model

Adding a custom endpoint: name, ID, protocol, base URL, API key and models.

1

Name it

Enter a Name. The ID (used in API calls) is derived from it and can’t change once added.
2

Pick the protocol and base URL

OpenAI-compatible or Anthropic-compatible, and the Base URL with its /v1 path, as the pods reach it: public https on xpander cloud (https://ai-gateway.example.com/v1), inside your network on Hybrid or Air-Gapped (http://vllm.ai-tools.svc.cluster.local:8000/v1). See address rules.
3

Pick a key

No key (local or unauthenticated server), Use an existing key or Create a new key.
4

List the models and save

Test connection reports Reachable and lists the models (a tools badge marks function calling), or Add model adds IDs. Choose Add provider. The models are on for your organization at once.
For a gateway your company already runs, see Your existing AI gateway. To check that Claude Code, Codex and OpenCode work with a model, choose Test model in its row’s menu. See Model testing.
To track spend, set a price per model with Edit model under Keys and models… (Price per 1M tokens (USD, optional)). Prices track spend only and are never charged. A model with no price reads as zero spend, so leave it at 0 if you don’t pay per token. The row’s menu also has Check now, Edit provider and Remove provider. New models that a vLLM, SGLang or NVIDIA NIM server starts serving are found with their context length and wait for an admin to approve them. Models already in use stay approved.
Every 2 minutes, the gateway runs the same agent check on one of each custom endpoint’s models (xpander’s own endpoints are not checked).A down endpoint shows a red banner in Models with the failed step, the cause, the fix and the last 24 checks. Open in Test model reruns everything. Model admins are notified in the app, by email and on Slack when it goes down and when it recovers.
Each custom model has a profile that reshapes every request to fit it. Open it with View profile (Edit profile if you may change it).A context-full refusal uses each CLI’s own error, so the CLI compacts as usual: OpenAI’s context_length_exceeded for Codex and OpenCode, Anthropic’s “prompt is too long” for Claude Code.Values come from four sources: Family default (built in for a model family), Learned by Test model (a check that found a fix saves it), Set by an admin (wins over the other two), and Discovered (the context length the endpoint reports, used when nothing else sets one). Family defaults: gpt-oss maps effort to reasoning_effort. Qwen, DeepSeek and GLM map it to chat_template_kwargs. Kimi and Nemotron strip thinking and effort, and Kimi accepts at most 128 function definitions. Each model’s context length comes from the endpoint’s model list (/v1/models, the max_model_len field on vLLM). The engine version comes from vLLM (/version), SGLang (/get_server_info) or NVIDIA NIM (/v1/version).
With No key (local or unauthenticated server), every call carries a placeholder, never a real key. Gateway callers name the model <ID>::<model>, such as internal_vllm::llama-3.3-70b. Check now and Test connection still work. Calls show in Logs at the price set with Edit model, and are not charged in credits.
On xpander cloud the endpoint must be public, so anyone who finds its address can use it. Protect the server with a key, or connect it from a Hybrid or Air-Gapped install, where it can stay private.
  • POST /v1/agents accepts only built-in providers in model_provider. A custom ID gets 422.
  • Pick a custom provider per conversation in the composer’s Harness and model popover (Claude Code, Codex, OpenCode).
  • A local model needs a context window well above the prompt xpander composes. Worked example: Air-Gapped: a worked vLLM setup.
  • With no /responses route on the endpoint, the gateway sends Codex calls as chat completions.

​
Model endpoints on xpander cloud

On xpander cloud, a model endpoint must be a public https address. On Hybrid or Air-Gapped it can be private, but never link-local or cloud metadata. Both are checked when saved and on every call.
This applies to Azure endpoints, OpenAI-compatible endpoints and model endpoints set on an agent.

​
Turn models on

In a provider’s row (or Keys and models…), tick the models the organization may use. Prices are USD per 1M tokens. Disable provider or Disable model stops it everywhere. A model blocked at the gateway shows Off (No one in the organization can use this model.) until Unblock at the gateway.

​
Who may use which models

Access panel for Amazon Bedrock listing Organization and the groups Default, Design, Engineering and Sales, each with All models ticked

Who may use it: the organization and each group, with which of the provider's models they may use. Shown with sample data.

Who may use it… in a provider’s or model’s menu sets what the Organization and each group may use: All models (every model the provider has on) or a list, within what the organization has on. A model outside a group’s list is hidden from its members’ pickers, and a run naming it is refused. Who can use what answers “why can’t this person use this model” By person, By group or By model, with the gateway’s own reason: Blocked for everyone, Group switched off, Not on the organization’s list or Not on the group’s list.

​
Aliases and fallbacks

An alias is a model name you define, such as default-coding. Clients send it as the model and the gateway calls its target, so changing the target moves every client at once. Add one with Add alias… in a model’s ⋯ menu. An overload or server error is retried on the same model, then on each fallback in order; a failure that will not change (invalid request, model off, spent budget) is not. Each fallback uses the key assigned for its own provider, and each attempt shows under Routing in Logs. Callers are still limited to the models they may use.
1

Start from a model

Open the provider, then the model’s ⋯ menu, and choose Add alias….
2

Name it and choose the target

Enter a Name: lowercase letters, digits, dots, dashes and underscores, starting with a letter or digit, 63 characters at most. A Description is optional. Points to starts on the model you chose. Pick any other model you turned on.
3

Add fallbacks and save

Under Fallbacks, tried in order, choose Add a fallback model for each model to try. The target can’t be a fallback, and each fallback appears once. Choose Add alias.
Names starting with xp- are reserved: Names starting with xp- are xpander’s built-in models; choose another name for: …
Each alias is a tag under its model. Aliases whose model is not listed join Other aliases. A grayed-out tag means the target is off. A warning dot means a fallback is off. Hover for details. Click a tag to edit it and choose Save alias, or use Delete alias in the card’s menu. Clients still sending a deleted alias get an error.People find aliases under Shortcuts your admin set up in AI models & keys, in the Model picker of Codex, OpenCode and SDK setups, and in /v1/models on the gateway address.

​
Whose key pays

A provider can hold several keys, such as one per team. In its card under AI Gateway, set the Organization default (or Automatic), assign keys in Give a group, person or agent its own key, and confirm with Check a person or agent. Each call uses the first key assigned to:
  1. the agent making the call
  2. the person making the call
  3. the person’s primary group
  4. the organization default for that provider
Whether a stored key or xpander’s key (billed in credits) pays at all depends on your license model. To rotate, assign a new key or replace the value in the vault; the next call uses it.
  • Rejected by the provider marks a key the provider refused. Used by the gateway marks the one in use.
  • Calls on keys xpander provides are charged in credits. Calls on a key you brought, agent or organization level, never are.
  • A provider with no key of your own shows the xpander key row (Metered in credits.) on xpander cloud, Installation credentials on a self-hosted install with a key in secrets.static, and Needs a key otherwise. Calls to a provider that needs a key are refused with a message that names it.
  • A deleted key stops being used from the next call.

​
Keys on a self-hosted install

The gateway is always on self-hosted, so every provider your agents use needs a key, the organization’s or the install’s, or calls are refused naming it. Hybrid agents never use xpander’s keys.
Every agent model call, including Bedrock and per-agent endpoints, goes through the gateway. The key comes from the first of: a credential set on one agent, the organization’s key for the provider (above), or the install’s keys in chart values secrets.static.<provider>ApiKey or a Kubernetes secret referenced with envFromSecretKeys. The keys a call can use are shown in the Providers table as Connected or Needs a key.
  • --reset-then-reuse-values needs Helm 3.14 or later. It keeps your values and applies the chart’s defaults for the rest.
  • Provider credentials live only on the Agent Controller. An Anthropic key there egresses to api.anthropic.com; to use your own gateway, set LLM_PROXY_ANTHROPIC_BASE_URL (Route the controller through your gateway).
  • A key named by an environment variable on agent-controller (custom provider, agent or organization default model) never reads the install’s own credentials. A name on the reserved list reads as no key (reserved key names).
  • Pods restart on their own when a secret changes. The xpander-static secret has a Helm keep policy: if a key is not picked up after an upgrade, patch the secret and restart xpander-agent-worker. Full mapping: Hybrid: managing LLM API keys.
  • Install-level defaults: global.llm.defaults.{provider,model} and global.llm.gateway.{provider,model,apiBase} in the chart values. Bedrock: global.llm.bedrock (Amazon Bedrock).

​
Defaults for new agents

The organization’s default provider picks a new agent’s harness, unless the create names one. Changing defaults does not touch existing agents.
A create that names a runtime, provider, model or harness settings is honored as written. Otherwise it is defaulted, the same way on cloud, Hybrid and Air-Gapped:
  1. The organization’s default LLM picks the CLI: No default, or a provider no CLI runs, falls through.
  2. The install’s default knobs (env on the agents and agent-controller services, the chart values below on Air-Gapped).
  3. The baked defaults: Claude Code on Anthropic direct with Fable 5.1 (claude-fable-5-1), Codex on gpt-6-astra, OpenCode on anthropic/claude-opus-5.
So a fresh Air-Gapped install with no default LLM creates Claude Code agents on Fable 5.1 over Anthropic direct, and needs an Anthropic key on the controller or Bedrock set as the default (with the opt-in above).Claude Code on Bedrock on Air-Gapped needs both harness defaults: global.harness.defaultProviders.claudeCode: amazon_bedrock and global.harness.defaultModels.claudeCode: <Bedrock model id>. Without them, the first turn fails with 503 no anthropic credential configured for harness agents, even when the organization’s default model is on Bedrock.

​
Next steps

Decide who pays for each call

License models and stored keys.

Connect Bedrock with an IAM role

No long-lived keys.