LLM Gateway on a self-hosted install

On Hybrid and Air-Gapped installs the gateway runs in your cluster. Model calls, provider keys and request records stay inside your environment; settings are managed in the Platform as on xpander cloud.

Turn it on

The gateway is on by default. To publish its host, set these values:

llm-gw:
  enabled: true
  ingress:
    enabled: true      # publish llm-gw.<your domain>
    internal: false    # true: serve the host on a private load balancer

Then upgrade the release. On an Air-Gapped install:

Mirror the gateway image first with image_sync.sh --manifest-out, which also pins it in your values. If the gateway is enabled without a pinned image, the render stops and names the value to set.
global.llmGw.enabled must match llm-gw.enabled. The installer writes both.

Value	Default	What it does
`llm-gw.enabled`	`true`	Runs the gateway
`llm-gw.ingress.enabled`	`false`	Publishes `llm-gw.<your domain>`; needs the chart-wide ingress on
`llm-gw.ingress.host`	`llm-gw.<your domain>`	Another host name
`llm-gw.ingress.internal`	`false`	Serves the host on a private load balancer (`internalClassName`, `internalAnnotations`)
`llm-gw.env`		`HTTPS_PROXY` and `NO_PROXY` on a cluster that reaches the internet through a proxy
`llm-gw.serviceAccountAnnotations`		Workload identity for the gateway’s own service account

Agents in your cluster reach the gateway by its in-cluster service name; the published host is for laptops, your apps and SDKs. Only the model paths are published (/v1/, /anthropic/, /gemini/, /openai/); health, metrics and everything else stay inside the cluster. On an Air-Gapped install, people point their coding tools at this host (Connect your tools). On a Hybrid install, people’s coding tools connect to xpander cloud instead (below), so publishing the host is needed only for your own apps and SDKs inside your network.

Long streams

A coding session can hold one response open for many minutes. The chart sets a 1500 second idle timeout on the ingress; set at least 1500 seconds on any load balancer in front of it (on an AWS Application Load Balancer, idle_timeout.timeout_seconds=1500).

Size

	Default
Servers	2 to 20, scaled on CPU
Each server	0.5 to 1 vCPU, 1 to 4 GiB memory
Database	None for the gateway itself; request records go to your install’s PostgreSQL

Plan the PostgreSQL storage by request volume and the retention you set in Platform > AI governance > Policies. See Sizing.

Keys and secrets

The chart generates the gateway’s internal key on install and keeps it on upgrade (Hybrid: Secret <release>-llm-gw; Air-Gapped: Secret xpander-llm-gw-keys). No caller ever uses it. To manage it yourself, set llm-gw.masterKey.existingSecret on Hybrid or create the Secret yourself with keygen off on Air-Gapped. On an ArgoCD install, either set it yourself or have ArgoCD ignore the generated Secret’s data, as in examples/argocd-application.yaml. The chart also generates the key that encrypts saved prompt and response text (Hybrid: Secret <release>-llm-gw-content, or set llm-gw.contentKey.existingSecret; Air-Gapped: keygen.overrides.LLM_GW_CONTENT_KEY). Keep it stable: saved text cannot be read after it changes. No provider key is set on the gateway. Provider keys stay in your vault and are added to each call (Providers and keys).

Network

From the gateway to	Why
Your model providers, or your OpenAI-compatible endpoints	The calls themselves
The xpander services in your cluster	Keys, settings and usage

With the chart’s network policy on, the gateway accepts traffic only on its serving port and can never reach the cloud metadata address. Egress is limited to public https addresses. To allow private model endpoints and private cloud endpoints, list them in llm-gw.networkPolicy (providerCidrs, cloudEndpointCidrs). A sealed install (no internet egress) uses only endpoints inside your network, and Bring your own key is the only license model. If your network requires its own certificate authority for a gateway or proxy, the LLM Gateway trusts it the same way the other services do (Air-gapped). Log destinations are sent from your install, so the collector or bucket you set only needs to be reachable from your cluster.

Hybrid specifics

Settings. Policies, models, keys and budgets are managed in the Platform and reach your install within seconds.
Agents. Agents on your install call your install’s gateway, on your stored keys and the credentials your install provides. xpander’s keys are never used.
People’s coding tools. Claude Code, Codex, OpenCode and SDKs on your people’s computers connect to xpander cloud at https://llm-gw.xpander.ai, with their personal xpander key (Connect your tools). Those calls run on the person’s own Claude plan or a provider key you stored in xpander, never on xpander’s keys or on credentials inside your install. A model whose provider has no key stored in xpander shows in the person’s model list with No key for this provider is stored in xpander, so tools on your computer can’t use these models. Agents in your own environment may still use them. Their prompt and response text is never kept in xpander cloud.
Records. Requests of agents on your install stay in your install: the Platform shows them when your browser can reach your environment directly, and totals for billing are sent with billing events. Requests from people’s coding tools are stored in your organization’s xpander cloud database and show in Logs.
Budgets. See Budgets on self-hosted installs.

Govern

LLM Gateway

Deploy

Integrations

Licensing and identity

Security and compliance

LLM Gateway on a self-hosted install

Turn it on

Long streams

Size

Keys and secrets

Network

Hybrid specifics

​Turn it on

​Long streams

​Size

​Keys and secrets

​Network

​Hybrid specifics

Turn it on

Long streams

Size

Keys and secrets

Network

Hybrid specifics