Skip to main content
On Hybrid and Air-Gapped installs, the gateway runs in your cluster. Your agents’ model calls, your provider keys and their request records stay inside your environment, and settings are managed in the Platform as on xpander cloud. On Hybrid, people’s coding tools use xpander cloud’s gateway instead, so those records are on xpander cloud (Hybrid specifics). Install-wide steps (images, ingress, network policy, keys) are on Hybrid and Air-Gapped; this page covers what is specific to the gateway.

​
Always on

Agents send every model call through the gateway, and it can’t be turned off: since Hybrid chart 0.22.5 and Air-Gapped chart 0.17.2, the chart ignores llm-gw.enabled. Agents use the in-cluster service at http://llm-gw:4000 on Air-Gapped and http://xpander-llm-gw:4000 on Hybrid with the release name xpander (release names), shown as Base URL on the Models page while no host is published. Preflight refuses an install with no provider for model calls: an organization default, a gateway provider, or a per-provider in-cluster endpoint must exist. Every provider your agents use needs a key, in Models or on the install (secrets.static), or calls are refused naming the missing key.

​
Publish a host for desktops, apps and SDKs

Set these values and upgrade the release:
A published host needs a DNS record and a certificate covering it; the chart reuses ingress.tls.secretName. Only the model paths are published (/v1/, /anthropic/, /gemini/, /openai/). On Air-Gapped, people point their coding tools at it (AI models & keys). On Hybrid their coding tools connect to xpander cloud, so the host is only for your own apps and SDKs. A coding session can hold one response open for many minutes. The chart sets a 1500-second idle timeout on the ingress. Set at least 1500 seconds on any load balancer in front of it (on an AWS Application Load Balancer, idle_timeout.timeout_seconds=1500).
While no gateway pod is ready, a fallback pod (llm-gw-unavailable on Air-Gapped, xpander-llm-gw-unavailable on Hybrid) answers with a JSON 503 that coding tools and SDKs retry.

​
Upgrading an existing install

Every upgrade starts the gateway: install.sh upgrade and any helm upgrade, including --reuse-values. A leftover llm-gw.enabled: false is ignored. If the gateway was off, the upgrade starts it and restarts agent-controller and api once. Before you upgrade:
  1. Mirror the image, if you mirror images. On Air-Gapped, run the current image_sync.sh --manifest-out first. It mirrors and pins the llm-gw and tool-repo-seed images. --icon-pack-tag no longer exists, so remove it from your scripts. If the image is not pinned, the render stops and names the value to set.
  2. Open the gateway’s network paths.
  3. Give every provider your agents use a key. Custom OpenAI-compatible endpoints keep working.
New agent runs wait for a healthy gateway. If its pods can’t start, such as when the image can’t be pulled, new runs are refused with “The xpander AI Gateway is unavailable right now”.

​
Check it

1

Check the AI Gateway pods

A healthy gateway shows two llm-gw pods (one on a single-node install.sh cluster), each 1/1 and Running.
2

Check the Agent Controller

3

Check the product

The Locations tab of Platform > Agents > Runtime shows the installation as Connected. A model call appears in the task’s usage and in Platform > AI governance > Logs.

​
Size, keys and network

The gateway runs 2 to 20 servers scaled on CPU (one on a single-node install.sh cluster), each 0.5 to 1 vCPU and 1 to 4 GiB. It has no database of its own: request records go to your install’s PostgreSQL, so size that for request volume and retention (Sizing). No provider key is set on the gateway; keys stay in your vault or in secrets.static and are added to each call.
Never delete or rotate the content key: saved text can’t be read after it changes. On Hybrid, both generated Secrets survive upgrade and uninstall. With ArgoCD, set both existingSecret values or have ArgoCD ignore the Secrets’ /data, as in examples/argocd-application.yaml.
The gateway reaches your model providers or OpenAI-compatible endpoints, plus the xpander services in your cluster and Redis. With the chart’s network policy on, it accepts traffic only on its serving port, can never reach the cloud metadata address, and egresses only to DNS, agent-controller, Redis and public https. List anything else in llm-gw.networkPolicy:Behind an outbound proxy, set HTTPS_PROXY and NO_PROXY in llm-gw.env. The gateway does not read global.env. A custom certificate authority is trusted as for the other services (Air-Gapped). A sealed install (no internet egress) uses only endpoints in your network, with Bring your own key as the only license model. Log destinations only need to be reachable from your cluster.

​
Hybrid specifics

  • Settings (policies, models, keys, budgets) are managed in the Platform and reach your install within seconds.
  • Agents on your install call your install’s gateway, on your stored keys and your install’s credentials, never xpander’s keys.
  • People’s coding tools connect to xpander cloud at https://llm-gw.xpander.ai with their personal xpander key (AI models & keys). They run on the person’s own Claude plan or a key you stored in xpander, never on xpander’s keys or your install’s credentials, and their prompt and response text is never kept in xpander cloud. A provider with no key stored in xpander shows No key for this provider is stored in xpander, so tools on your computer can’t use these models. Agents in your own environment may still use them.
  • Records of agents on your install stay there and show in the Platform when your browser can reach your environment. Totals for billing go with billing events. Records from people’s coding tools live in xpander cloud and show in Logs.
  • Budgets: see Budgets on self-hosted installs.

​
Next steps

Add provider keys

A key for every provider.

Run a local model air-gapped

Local LLM inference.

Plan the install's size

PostgreSQL for volume and retention.