> ## Documentation Index
> Fetch the complete documentation index at: https://docs.xpander.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Upgrade and operate Hybrid

> Upgrade a Hybrid install to the current chart, drive the installer from scripts and CI, and uninstall.

Watch the [xpander releases feed](https://github.com/xpander-ai/releases) (Releases page / Atom feed), then run the `upgrade` subcommand:

```bash theme={"dark"}
curl -fsSL https://charts.xpander.ai/install.sh | bash -s -- upgrade
# or, pinned:
curl -fsSL https://charts.xpander.ai/install.sh | bash -s -- upgrade --chart-version <X.Y.Z>
```

## Upgrading

`install.sh upgrade`:

* finds the release (default `xpander` in namespace `xpander`, override with `--release` / `--namespace`),
* **infers its edition** from the installed chart: you never pass `--edition` to an upgrade,
* **reuses the values the release was installed with** (`helm get values`), plus anything extra you pass (`--set`, `--values`),
* shows current → target version with a link to the release notes, and asks before changing anything,
* always runs with `--atomic`: a failed upgrade (including a failed hook) rolls back automatically and the previous revision keeps serving.

Manual-Helm equivalent (Helm 3.14 or later):

```bash theme={"dark"}
helm repo update
helm upgrade xpander xpander/xpander --namespace xpander --reset-then-reuse-values
```

`--reset-then-reuse-values` keeps the values you set and applies the new chart's defaults for everything else. Plain `--reuse-values` replays the previous release's values, including the old chart's defaults, so it misses defaults that changed (see below).

### Upgrading to the current chart

The current Hybrid chart is `xpander` 0.22.5 (release v2026.10.22). Database migrations run automatically. Upgrade with Helm 3.14 or later:

```bash theme={"dark"}
helm repo add xpander https://charts.xpander.ai && helm repo update xpander
helm upgrade xpander xpander/xpander --version 0.22.5 -n <namespace> --reset-then-reuse-values --wait --timeout 30m
```

**Before you upgrade from a chart between 0.22.0 and 0.22.4:** the changes below arrived across charts 0.22.1 to 0.22.5. Each one happens on the first upgrade to the chart that brings it, so skip the ones your current chart already has. On 0.22.3, for example, only the last two rows still apply.

| Change | Chart that brings it |
| - | - |
| In-cluster Postgres on the pgvector build, with the pre-upgrade check | 0.22.1 |
| `BRAIN_INGEST_API_KEY` in the generated Secret | 0.22.3 |
| `envFromSecret` and `envFromConfigMap` render | 0.22.4 |
| The LLM Gateway is always on | 0.22.5 |

> [accordion] What each change needs (0.22.1 to 0.22.5)
>
> * **In-cluster Postgres moves to the pgvector build (0.22.1).** The `postgres` StatefulSet runs `pgvector/pgvector:0.8.7-pg15-trixie` instead of `postgres:15`: PostgreSQL 15.19, the same major version, on the same volume. If you use `global.imageRegistry`, mirror `docker.io/pgvector/pgvector:0.8.7-pg15-trixie` first. Take a volume snapshot of `postgres-data-xpander-postgres-0`, or set `postgres.upgradeGuard.backup.pvcName` to an existing PVC with room for a full dump. Postgres restarts once. If you already run 0.22.1 or later, Postgres is on pgvector and this step is done.
> * **A pinned Postgres image stays on pgvector (0.22.1).** If your values set `images.postgres`, keep it on a pgvector build of PostgreSQL 15. A pre-upgrade check refuses the upgrade when the major version would change.
> * **A new key in the generated Secret (0.22.3).** `BRAIN_INGEST_API_KEY` is added automatically and kept from then on. No action needed.
> * **Environment from your own Secrets and ConfigMaps renders (0.22.4).** `global.envFromSecret`, `global.envFromConfigMap` and the per-component `envFromSecret` and `envFromConfigMap` lists now render as the container's `envFrom`. Earlier charts failed the render when they were set. Installs that never set them render the same as before.
> * **The LLM Gateway is always on (0.22.5).** The chart ignores `llm-gw.enabled`. If your install had `llm-gw.enabled: false`, mirror the `llm-gw` image if you mirror images, make sure every model provider your agents use has a key, and open the gateway's network paths ([LLM Gateway](https://docs.xpander.ai/self-hosted/hybrid-llm-gateway.md)). The upgrade starts the gateway pods and restarts agent-controller and api once. Then check `kubectl -n <namespace> get pods -l app=llm-gw`. If the gateway was already on, nothing changes.
> * **Per-agent MCP URLs move to the MCP host (release change, no chart change).** MCP clients that used a per-agent MCP URL connect to `https://mcp.<DOMAIN>` instead. Per-agent MCP URLs are no longer served. The API's own MCP endpoint, `https://api.<DOMAIN>/mcp/` ([Model Context Protocol](https://docs.xpander.ai/api-reference/mcp.md)), does not change, so clients set up on it keep working. The chart has served `mcp.<DOMAIN>` from its ingress since 0.22.0. There is no value to set.

> [accordion] The Postgres upgrade, step by step
>
> A pre-upgrade Job, `xpander-postgres-upgrade-guard`, refuses the upgrade when the running major version differs from the image's, and lists every database. With `postgres.upgradeGuard.backup.pvcName` set, it first writes a `pg_dumpall` to that PVC, and refuses the upgrade if the dump cannot be taken in full. The dump must finish within `postgres.upgradeGuard.timeoutSeconds` (default 1500) and within the `--timeout` of the upgrade, so raise both for a large database. After the upgrade, the `xpander-postgres-pgvector` Job adds the `vector` extension to every database. It never fails the release, and its log ends with one status line:
>
> ```bash theme={"dark"}
> kubectl -n <namespace> logs job/xpander-postgres-pgvector
> ```
>
> To go back, `helm rollback <release> <previous revision>` returns the pod to `postgres:15` on the same volume with all its data.

> [accordion] Coming from a chart before 0.22.0
>
> These changes also apply:
>
> * **The LLM Gateway runs.** Before you upgrade, mirror the `llm-gw` image if you mirror images, make sure every provider your agents use has a key, and open any network path the gateway needs ([LLM Gateway](https://docs.xpander.ai/self-hosted/hybrid-llm-gateway.md)). agent-controller and api restart once to pick up the gateway's address. After the upgrade, check `kubectl -n <namespace> get pods -l app=llm-gw` ([healthy result](https://docs.xpander.ai/self-hosted/hybrid-llm-gateway.md#health)).
>
> * **Argo CD.** Add `ignoreDifferences` on `/data` for `xpander-llm-gw` and `xpander-llm-gw-content`, or set `llm-gw.masterKey.existingSecret` and `llm-gw.contentKey.existingSecret`.
>
> * **Harness agents run only on the fleet executors.** There is no pod per agent. When no executor is up, a turn waits in the queue. The chart always keeps at least one executor: `fleet.replicas: 0` (or blank) renders one, and with `fleet.keda.enabled`, `fleet.keda.maxReplicas: 0` holds one. `agentContainers.harness` (`enabled`, `resources`, `storage`, `shm`) is ignored and can be removed from your values. `images.agentHarness` stays. It is the image every executor runs.
>
> * **The release Role reads pods but no longer writes them.** agent-controller keeps get, list and watch on pods and still manages the Deployments, Services, volumes, Secrets and Jobs of agent workspaces. If your own automation used the release ServiceAccount to create, update or delete pods, add the rule back:
>
>   ```yaml theme={"dark"}
>   rbac:
>     additionalRules:
>       - apiGroups: [""]
>         resources: ["pods"]
>         verbs: ["create", "update", "patch", "delete"]
>   ```
>
> * **Fleet agents reach private networks like any other pod.** `fleet.networkPolicy.exceptCidrs` now defaults to `["169.254.0.0/16"]` and `exceptCidrsV6` to `["fe80::/10"]`, so agents and the MCP endpoints they call reach `10/8`, `172.16/12`, `192.168/16` and in-cluster services. Cloud metadata stays blocked. `--reset-then-reuse-values` picks this up. With `--reuse-values`, pass `--set-json 'fleet.networkPolicy.exceptCidrs=["169.254.0.0/16"]' --set-json 'fleet.networkPolicy.exceptCidrsV6=["fe80::/10"]'`. An install that set `exceptCidrs` itself keeps its own list.
>
> * **Executor nodes need Landlock, or `fleet.landlock.enabled: false`.** See [Node prerequisites](https://docs.xpander.ai/self-hosted/sizing-runtime.md#node-prerequisites).
>
> * **Model key names and endpoints.** Environment-variable key names that match the install's own credentials read as no key, and model endpoints on metadata addresses are refused ([details](https://docs.xpander.ai/self-hosted/hybrid-llm-keys.md#key-names-read-from-an-environment-variable)). Check the key names on your agents, custom providers and the organization's default model before you upgrade.
>
> * **Custom-container agents** move to the standard agent runtime during the upgrade.
>
> `--reuse-values` with the new defaults:
>
> ```bash theme={"dark"}
> helm upgrade xpander xpander/xpander --namespace xpander --reuse-values \
>   --set-json 'fleet.networkPolicy.exceptCidrs=["169.254.0.0/16"]' \
>   --set-json 'fleet.networkPolicy.exceptCidrsV6=["fe80::/10"]'
> ```

What else to expect from the current chart line:

* **PostgreSQL no longer restarts on every chart release.** The config checksum hashes the ConfigMap data only, so a release whose Postgres configuration is byte-identical leaves the StatefulSet alone. A real `postgres.*` change still rolls it.
* **The fleet is always rendered, and `--atomic` waits for it.** A running turn holds the roll for up to the drain window (`fleet.executor.drainSeconds`, 3600 s). The idle guard holds the executor at zero replicas on a cluster without its StorageClass, so the upgrade completes anyway. If you drive `helm upgrade --wait` by hand, give `--timeout` at least the drain window or upgrade in a quiet period. Details on [Runtime and node reference](https://docs.xpander.ai/self-hosted/sizing-runtime.md#safe-rollout-drain-and-upgrades).

## Automation and CI

The installer is built to be driven by scripts, CI pipelines, and coding agents:

* **Every prompt has a flag.** With `--non-interactive` nothing is read from the terminal, even when one exists: missing required values exit 2 with the usage and a completed rerun line rather than blocking. It implies `--yes`. All flags: [Installer flags reference](https://docs.xpander.ai/self-hosted/hybrid-install.md#installer-flags-reference).
* **Stable progress output.** Progress is one line per transition in a grep-friendly shape, `[3/6] chart-source: ok`, and `--quiet` reduces output to phase lines and the outcome only. Color is emitted only when stdout is a terminal.
* **`status` is the poll target.** A read-only health report: edition, chart, release status, hook jobs, workload readiness, and URLs. `--json` emits one JSON object.

**Hybrid, end to end, no terminal:**

```bash theme={"dark"}
bash install.sh --non-interactive --quiet \
  --edition hybrid --org <uuid> --env <uuid> --dm-api-key <key> \
  --domain xpander.example.com --openai-key <key>
```

**Poll until healthy:**

```bash theme={"dark"}
until bash install.sh status --quiet; do sleep 30; done
```

When a scripted run fails, the exit code plus the last `FIX:`/`RERUN:` lines tell you what to repair. See [Troubleshooting](https://docs.xpander.ai/self-hosted/hybrid-troubleshooting.md).

> [accordion] Exit codes
>
> | Code | Meaning |
> | - | - |
> | `0` | Success |
> | `1` | Install or upgrade failure (`status`: degraded) |
> | `2` | Usage error, or a required flag missing with no terminal |
> | `3` | `status`: no such release installed |
> | `4` | Preflight failure (missing helm/kubectl, unreachable cluster, version too old) |
> | `5` | External data-tier probe failed and `--on-probe-failure` was `abort` (air-gap external data tier) |

## Uninstalling

```bash theme={"dark"}
# Remove the Helm release
helm uninstall xpander --namespace xpander

# Clean up the namespace (optional)
kubectl delete namespace xpander
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.