> ## Documentation Index
> Fetch the complete documentation index at: https://docs.xpander.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# IAM Best Practices

> Three-tier IAM role architecture for self-hosted xpander — base role, connector roles, and cross-account trust for least-privilege access

## Architecture Overview

Xpander self-hosted uses a three-tier IAM role architecture for connector authentication:

```
┌─────────────────────────────────────────────────────┐
│  EKS Pod (xpander service account)                  │
│                                                     │
│  Pod Identity ──► Base Role (no permissions)        │
│                       │                             │
│                       ├──► Redshift Connector Role  │
│                       ├──► EKS Connector Role       │
│                       ├──► S3 Connector Role        │
│                       └──► [Any Future Connector]   │
└─────────────────────────────────────────────────────┘
```

| Role                    | Purpose                                                                                                   | Permissions                                                       |
| ----------------------- | --------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| **Base Role**           | Attached to pods via EKS Pod Identity. Has no direct permissions — only assumes connector-specific roles. | `sts:AssumeRole` + `sts:TagSession` on each connector role        |
| **Connector Roles**     | One per data source/service. Contains only the permissions needed for that connector.                     | Service-specific (e.g., `redshift-data:*`, `eks:*`) + self-assume |
| **Cross-Account Trust** | Each connector role can optionally trust the xpander platform for cloud-managed operations.               | Trust with `ExternalId` condition                                 |

### Why Three Tiers?

1. **Least privilege** — Each connector only has access to its specific service. A Redshift connector can't access EKS and vice versa.
2. **Auditability** — CloudTrail shows exactly which connector role accessed which resource. No shared credentials.
3. **Independent lifecycle** — Add or remove connectors without touching other roles. Revoke one connector's access without affecting others.
4. **Role chaining** — The AI Gateway assumes the appropriate connector role per request. This is standard AWS role chaining, fully supported by xpander.

<Info>
  If you're using a single IAM role for all connectors (the setup from [AWS Operator Setup](/self-hosted/aws-operator)), the instructions on this page show how to split that into per-connector roles for better isolation and auditability.
</Info>

***

## Step 1: Base Role

The base role is the only role attached to pods. It has no service permissions — it can only assume other roles.

```bash theme={"dark"}
aws iam create-role \
  --role-name xpander-base-role \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": { "Service": "pods.eks.amazonaws.com" },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    }]
  }'
```

Attach the assume permissions (update this list as you add connectors):

```bash theme={"dark"}
aws iam put-role-policy \
  --role-name xpander-base-role \
  --policy-name AssumeConnectorRoles \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": ["sts:AssumeRole", "sts:TagSession"],
      "Resource": [
        "arn:aws:iam::<ACCOUNT_ID>:role/xpander-redshift-access",
        "arn:aws:iam::<ACCOUNT_ID>:role/xpander-eks-connector"
      ]
    }]
  }'
```

Associate with the xpander service account via Pod Identity:

```bash theme={"dark"}
aws eks create-pod-identity-association \
  --cluster-name <CLUSTER_NAME> \
  --namespace xpander \
  --service-account xpander \
  --role-arn arn:aws:iam::<ACCOUNT_ID>:role/xpander-base-role \
  --region <REGION>
```

***

## Step 2: Connector Roles

Each connector role needs a trust policy with four statements:

| Statement                    | Purpose                                              | ExternalId Required? |
| ---------------------------- | ---------------------------------------------------- | -------------------- |
| `PodIdentity`                | Direct pod access (fallback)                         | No                   |
| `CrossAccountWithExternalId` | xpander cloud platform operations                    | Yes                  |
| `SelfAssume`                 | AI Gateway internal re-assume for credential refresh | No                   |
| `AllowFromBaseRole`          | Role chaining from the Pod Identity base role        | No                   |

### Trust Policy Template

```json theme={"dark"}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "PodIdentity",
      "Effect": "Allow",
      "Principal": { "Service": "pods.eks.amazonaws.com" },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    },
    {
      "Sid": "CrossAccountWithExternalId",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<ACCOUNT_ID>:root"
      },
      "Action": ["sts:AssumeRole", "sts:TagSession"],
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<ORGANIZATION_ID>"
        }
      }
    },
    {
      "Sid": "SelfAssume",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<ACCOUNT_ID>:role/<THIS_ROLE_NAME>"
      },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    },
    {
      "Sid": "AllowFromBaseRole",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<ACCOUNT_ID>:role/xpander-base-role"
      },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    }
  ]
}
```

<Warning>
  The `SelfAssume` and `AllowFromBaseRole` statements must **not** have an `ExternalId` condition. The AI Gateway's internal `AssumeRole` calls do not pass an external ID. If you add an ExternalId condition to these statements, the connector will fail with `AccessDenied`.
</Warning>

### Permission Policies Per Connector

Each connector role needs its own service permissions plus self-assume.

#### Self-Assume + Session Tagging (required for all connector roles)

```bash theme={"dark"}
aws iam put-role-policy \
  --role-name <CONNECTOR_ROLE> \
  --policy-name SelfAssumeAndTagSession \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": ["sts:AssumeRole", "sts:TagSession"],
      "Resource": "arn:aws:iam::<ACCOUNT_ID>:role/<CONNECTOR_ROLE>"
    }]
  }'
```

<AccordionGroup>
  <Accordion title="Redshift Connector">
    ```bash theme={"dark"}
    aws iam put-role-policy \
      --role-name xpander-redshift-access \
      --policy-name RedshiftCredentials \
      --policy-document '{
        "Version": "2012-10-17",
        "Statement": [{
          "Effect": "Allow",
          "Action": [
            "redshift:GetClusterCredentials",
            "redshift:GetClusterCredentialsWithIAM",
            "redshift:DescribeClusters"
          ],
          "Resource": [
            "arn:aws:redshift:<REGION>:<ACCOUNT_ID>:cluster:<CLUSTER_ID>",
            "arn:aws:redshift:<REGION>:<ACCOUNT_ID>:dbname:<CLUSTER_ID>/<DB_NAME>",
            "arn:aws:redshift:<REGION>:<ACCOUNT_ID>:dbuser:<CLUSTER_ID>/*"
          ]
        }]
      }'

    aws iam put-role-policy \
      --role-name xpander-redshift-access \
      --policy-name RedshiftDataAPI \
      --policy-document '{
        "Version": "2012-10-17",
        "Statement": [{
          "Effect": "Allow",
          "Action": [
            "redshift-data:ExecuteStatement",
            "redshift-data:GetStatementResult",
            "redshift-data:DescribeStatement",
            "redshift-data:ListStatements",
            "redshift-data:CancelStatement",
            "redshift-data:BatchExecuteStatement",
            "redshift-data:ListDatabases",
            "redshift-data:ListSchemas",
            "redshift-data:ListTables",
            "redshift-data:DescribeTable"
          ],
          "Resource": "*"
        }]
      }'
    ```
  </Accordion>

  <Accordion title="EKS Connector">
    ```bash theme={"dark"}
    aws iam put-role-policy \
      --role-name xpander-eks-connector \
      --policy-name EKSAccess \
      --policy-document '{
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Action": [
              "eks:DescribeCluster",
              "eks:ListClusters",
              "eks:ListNodegroups",
              "eks:DescribeNodegroup",
              "eks:ListInsights",
              "eks:DescribeInsight",
              "eks:AccessKubernetesApi"
            ],
            "Resource": "*"
          },
          {
            "Effect": "Allow",
            "Action": [
              "logs:GetLogEvents",
              "logs:FilterLogEvents",
              "logs:DescribeLogGroups",
              "logs:DescribeLogStreams"
            ],
            "Resource": "*"
          }
        ]
      }'
    ```

    The EKS connector also needs Kubernetes RBAC:

    ```bash theme={"dark"}
    aws eks create-access-entry \
      --cluster-name <CLUSTER_NAME> \
      --principal-arn arn:aws:iam::<ACCOUNT_ID>:role/xpander-eks-connector \
      --type STANDARD \
      --region <REGION>

    aws eks associate-access-policy \
      --cluster-name <CLUSTER_NAME> \
      --principal-arn arn:aws:iam::<ACCOUNT_ID>:role/xpander-eks-connector \
      --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy \
      --access-scope type=cluster \
      --region <REGION>
    ```
  </Accordion>
</AccordionGroup>

***

## Step 3: Configure in xpander UI

In the connector settings, set the **IAM Role ARN** to the connector-specific role (not the base role):

| Connector | Role ARN                                                 |
| --------- | -------------------------------------------------------- |
| Redshift  | `arn:aws:iam::<ACCOUNT_ID>:role/xpander-redshift-access` |
| EKS       | `arn:aws:iam::<ACCOUNT_ID>:role/xpander-eks-connector`   |

***

## Adding a New Connector

When adding a new AWS connector (e.g., S3, DynamoDB, Athena):

<Steps>
  <Step title="Create the connector role">
    Use the [trust policy template](#trust-policy-template) above.
  </Step>

  <Step title="Attach service-specific permissions">
    Add the service permissions + the [self-assume policy](#self-assume--session-tagging-required-for-all-connector-roles).
  </Step>

  <Step title="Update the base role">
    Add the new role ARN to the `AssumeConnectorRoles` policy on the base role.
  </Step>

  <Step title="Configure in xpander UI">
    Set the connector's IAM Role ARN.
  </Step>

  <Step title="Restart the AI Gateway">
    ```bash theme={"dark"}
    kubectl rollout restart deployment xpander-ai-gateway -n xpander
    ```
  </Step>
</Steps>

No Pod Identity changes needed — the base role association stays the same.

***

## Role Creation Order

IAM roles can't reference themselves during creation. Follow this order:

1. Create the role with only `PodIdentity` + `CrossAccountWithExternalId` trust statements
2. Wait 10 seconds for IAM propagation
3. Update the trust policy to add `SelfAssume` + `AllowFromBaseRole` statements

```bash theme={"dark"}
# Step 1: Create without self-reference
aws iam create-role --role-name <ROLE> --assume-role-policy-document file://initial-trust.json

# Step 2: Wait for propagation
sleep 10

# Step 3: Update with self-assume
aws iam update-assume-role-policy --role-name <ROLE> --policy-document file://full-trust.json
```

***

## Troubleshooting

<AccordionGroup>
  <Accordion title="AccessDenied: sts:TagSession">
    **Cause:** Missing `sts:TagSession` in either the trust policy or the permission policy.

    **Fix:** Ensure both:

    * Trust policy allows `sts:TagSession` from the caller
    * Permission policy includes `sts:TagSession` on the role's own ARN
  </Accordion>

  <Accordion title="AccessDenied: sts:AssumeRole (self-assume)">
    **Cause:** The `SelfAssume` trust statement has an `ExternalId` condition, or the self-assume permission policy is missing.

    **Fix:** The `SelfAssume` and `AllowFromBaseRole` statements must NOT have conditions. The AI Gateway does not pass an ExternalId during internal role operations.
  </Accordion>

  <Accordion title="AccessDenied: sts:AssumeRole (from base role)">
    **Cause:** Base role doesn't have permission to assume the connector role, or the connector role's trust policy doesn't list the base role.

    **Fix:** Check both sides:

    * Base role permission policy lists the connector role ARN
    * Connector role trust policy has `AllowFromBaseRole` statement
  </Accordion>

  <Accordion title="Connector works in cloud but not self-hosted">
    **Cause:** In cloud mode, the xpander platform assumes the role directly with the ExternalId. In self-hosted mode, the AI Gateway chains through the base role without an ExternalId.

    **Fix:** Add the `AllowFromBaseRole` trust statement to the connector role.
  </Accordion>
</AccordionGroup>

***

## Security Recommendations

1. **Scope resources** — Use specific ARNs in permission policies, not `"Resource": "*"` where possible
2. **Separate roles per data source** — Don't combine Redshift + EKS permissions in one role
3. **Rotate ExternalId** — The organization ID is used as ExternalId. If compromised, rotate it in the xpander platform
4. **Monitor with CloudTrail** — Each connector role appears separately in CloudTrail, making it easy to audit which connector accessed which resource
5. **Tag roles** — Tag all roles with `Environment`, `Project`, `ManagedBy` for governance
6. **Restrict base role** — The base role should ONLY have `sts:AssumeRole` + `sts:TagSession`. Never attach service permissions directly to it