IAM Best Practices

This page covers skill access to AWS services (Redshift, Athena, S3, EKS). It does not cover model access: Amazon Bedrock is called with the pods’ own credentials, so bedrock:InvokeModel* and bedrock:Converse* belong on the role bound directly to the pods, not on a skill role behind the base role’s sts:AssumeRole chain. See Amazon Bedrock without a key and, for Hybrid, Amazon Bedrock: keyless IAM or API key.

Architecture Overview

xpander self-hosted uses a three-tier IAM role architecture for skill authentication:

┌─────────────────────────────────────────────────────┐
│  EKS Pod (xpander service account)                  │
│                                                     │
│  Pod Identity ──► Base Role (no permissions)        │
│                       │                             │
│                       ├──► Redshift Skill Role      │
│                       ├──► EKS Skill Role           │
│                       ├──► S3 Skill Role            │
│                       └──► [Any Future Skill]       │
└─────────────────────────────────────────────────────┘

Role	Purpose	Permissions
Base Role	Attached to pods via EKS Pod Identity. Has no direct permissions - only assumes skill-specific roles.	`sts:AssumeRole` + `sts:TagSession` on each skill role
Skill Roles	One per data source/service. Contains only the permissions needed for that skill.	Service-specific (e.g., `redshift-data:`, `eks:`) + self-assume
Cross-Account Trust	Each skill role can optionally trust xpander for cloud-managed operations.	Trust with `ExternalId` condition

Why Three Tiers?

Least privilege - Each skill only has access to its specific service. The Redshift skill can’t access EKS and vice versa.
Auditability - CloudTrail shows exactly which skill role accessed which resource. No shared credentials.
Independent lifecycle - Add or remove skills without touching other roles. Revoke one skill’s access without affecting others.
Role chaining - The agent-controller’s skill relay (agent-controller) assumes the appropriate skill role per request. This is standard AWS role chaining, fully supported by xpander.

If you’re using a single IAM role for all skills (the setup from AWS Operator Setup), the instructions on this page show how to split that into per-skill roles for better isolation and auditability.

Step 1: Base Role

The base role is the only role attached to pods. It has no service permissions - it can only assume other roles.

aws iam create-role \
  --role-name xpander-base-role \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": { "Service": "pods.eks.amazonaws.com" },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    }]
  }'

Attach the assume permissions to the xpander-base-role role, as an inline policy named AssumeConnectorRoles (update the Resource list as you add skills):

aws iam put-role-policy \
  --role-name xpander-base-role \
  --policy-name AssumeConnectorRoles \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": ["sts:AssumeRole", "sts:TagSession"],
      "Resource": [
        "arn:aws:iam::<ACCOUNT_ID>:role/xpander-redshift-access",
        "arn:aws:iam::<ACCOUNT_ID>:role/xpander-eks-connector"
      ]
    }]
  }'

Associate with the xpander service account via Pod Identity:

# one association per ServiceAccount the chart runs pods as: agent-controller and default
for sa in agent-controller default; do
  aws eks create-pod-identity-association \
    --cluster-name <CLUSTER_NAME> \
    --namespace xpander \
    --service-account "$sa" \
    --role-arn arn:aws:iam::<ACCOUNT_ID>:role/xpander-base-role \
    --region <REGION>
done

Step 2: Skill Roles

Each skill role needs a trust policy with four statements:

Statement	Purpose	ExternalId Required?
`PodIdentity`	Direct pod access (fallback)	No
`CrossAccountWithExternalId`	xpander cloud operations	Yes
`SelfAssume`	Skill relay internal re-assume for credential refresh	No
`AllowFromBaseRole`	Role chaining from the Pod Identity base role	No

Trust Policy Template

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "PodIdentity",
      "Effect": "Allow",
      "Principal": { "Service": "pods.eks.amazonaws.com" },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    },
    {
      "Sid": "CrossAccountWithExternalId",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<ACCOUNT_ID>:root"
      },
      "Action": ["sts:AssumeRole", "sts:TagSession"],
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<ORGANIZATION_ID>"
        }
      }
    },
    {
      "Sid": "SelfAssume",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<ACCOUNT_ID>:role/<THIS_ROLE_NAME>"
      },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    },
    {
      "Sid": "AllowFromBaseRole",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<ACCOUNT_ID>:role/xpander-base-role"
      },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    }
  ]
}

The SelfAssume and AllowFromBaseRole statements must not have an ExternalId condition. The skill relay’s internal AssumeRole calls do not pass an external ID. If you add an ExternalId condition to these statements, the skill will fail with AccessDenied.

Permission Policies Per Skill

Each skill role needs its own service permissions plus self-assume.

Self-Assume + Session Tagging (required for all skill roles)

aws iam put-role-policy \
  --role-name <CONNECTOR_ROLE> \
  --policy-name SelfAssumeAndTagSession \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": ["sts:AssumeRole", "sts:TagSession"],
      "Resource": "arn:aws:iam::<ACCOUNT_ID>:role/<CONNECTOR_ROLE>"
    }]
  }'

Redshift Skill

aws iam put-role-policy \
  --role-name xpander-redshift-access \
  --policy-name RedshiftCredentials \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": [
        "redshift:GetClusterCredentials",
        "redshift:GetClusterCredentialsWithIAM",
        "redshift:DescribeClusters"
      ],
      "Resource": [
        "arn:aws:redshift:<REGION>:<ACCOUNT_ID>:cluster:<CLUSTER_ID>",
        "arn:aws:redshift:<REGION>:<ACCOUNT_ID>:dbname:<CLUSTER_ID>/<DB_NAME>",
        "arn:aws:redshift:<REGION>:<ACCOUNT_ID>:dbuser:<CLUSTER_ID>/*"
      ]
    }]
  }'

aws iam put-role-policy \
  --role-name xpander-redshift-access \
  --policy-name RedshiftDataAPI \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": [
        "redshift-data:ExecuteStatement",
        "redshift-data:GetStatementResult",
        "redshift-data:DescribeStatement",
        "redshift-data:ListStatements",
        "redshift-data:CancelStatement",
        "redshift-data:BatchExecuteStatement",
        "redshift-data:ListDatabases",
        "redshift-data:ListSchemas",
        "redshift-data:ListTables",
        "redshift-data:DescribeTable"
      ],
      "Resource": "*"
    }]
  }'

EKS Skill

aws iam put-role-policy \
  --role-name xpander-eks-connector \
  --policy-name EKSAccess \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": [
          "eks:DescribeCluster",
          "eks:ListClusters",
          "eks:ListNodegroups",
          "eks:DescribeNodegroup",
          "eks:ListInsights",
          "eks:DescribeInsight",
          "eks:AccessKubernetesApi"
        ],
        "Resource": "*"
      },
      {
        "Effect": "Allow",
        "Action": [
          "logs:GetLogEvents",
          "logs:FilterLogEvents",
          "logs:DescribeLogGroups",
          "logs:DescribeLogStreams"
        ],
        "Resource": "*"
      }
    ]
  }'

The EKS skill also needs Kubernetes RBAC:

aws eks create-access-entry \
  --cluster-name <CLUSTER_NAME> \
  --principal-arn arn:aws:iam::<ACCOUNT_ID>:role/xpander-eks-connector \
  --type STANDARD \
  --region <REGION>

aws eks associate-access-policy \
  --cluster-name <CLUSTER_NAME> \
  --principal-arn arn:aws:iam::<ACCOUNT_ID>:role/xpander-eks-connector \
  --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy \
  --access-scope type=cluster \
  --region <REGION>

Step 3: Configure in xpander UI

In the skill’s connection settings, set the IAM Role ARN to the skill-specific role (not the base role):

Skill	Role ARN
Redshift	`arn:aws:iam::<ACCOUNT_ID>:role/xpander-redshift-access`
EKS	`arn:aws:iam::<ACCOUNT_ID>:role/xpander-eks-connector`

Adding a New Skill

When adding a new AWS skill (e.g., S3, DynamoDB, Athena):

Create the skill role

Use the trust policy template above.

Attach service-specific permissions

Add the service permissions + the self-assume policy.

Update the base role

Add the new role ARN to the AssumeConnectorRoles policy on the base role.

Configure in xpander UI

Set the skill’s IAM Role ARN.

Restart the skill relay

kubectl rollout restart deployment xpander-ai-gateway -n xpander

No Pod Identity changes needed - the base role association stays the same.

Role Creation Order

IAM roles can’t reference themselves during creation. Follow this order:

Create the role with only PodIdentity + CrossAccountWithExternalId trust statements
Wait 10 seconds for IAM propagation
Update the trust policy to add SelfAssume + AllowFromBaseRole statements

# Step 1: Create without self-reference
aws iam create-role --role-name <ROLE> --assume-role-policy-document file://initial-trust.json

# Step 2: Wait for propagation
sleep 10

# Step 3: Update with self-assume
aws iam update-assume-role-policy --role-name <ROLE> --policy-document file://full-trust.json

Troubleshooting

AccessDenied: sts:TagSession

Cause: Missing sts:TagSession in either the trust policy or the permission policy.Fix: Ensure both:

Trust policy allows sts:TagSession from the caller
Permission policy includes sts:TagSession on the role’s own ARN

AccessDenied: sts:AssumeRole (self-assume)

Cause: The SelfAssume trust statement has an ExternalId condition, or the self-assume permission policy is missing.Fix: The SelfAssume and AllowFromBaseRole statements must NOT have conditions. The skill relay does not pass an ExternalId during internal role operations.

AccessDenied: sts:AssumeRole (from base role)

Cause: Base role doesn’t have permission to assume the skill role, or the skill role’s trust policy doesn’t list the base role.Fix: Check both sides:

Base role permission policy lists the skill role ARN
Skill role trust policy has AllowFromBaseRole statement

Skill works in cloud but not self-hosted

Cause: In cloud mode, xpander assumes the role directly with the ExternalId. In self-hosted mode, the skill relay chains through the base role without an ExternalId.Fix: Add the AllowFromBaseRole trust statement to the skill role.

Security Recommendations

Scope resources - Use specific ARNs in each skill role’s permission policy, not "Resource": "*" where possible
Separate roles per data source - Don’t combine Redshift + EKS permissions in one role
Rotate ExternalId - The organization ID is used as ExternalId. If it is compromised, contact xpander to have it rotated
Monitor with CloudTrail - Each skill role appears separately in CloudTrail, making it easy to audit which skill accessed which resource
Tag roles - Tag all roles with Environment, Project, ManagedBy for governance
Restrict base role - The base role should ONLY have sts:AssumeRole + sts:TagSession. Never attach service permissions directly to it

Govern

Deploy

Integrations

Licensing and identity

Security and compliance

Architecture Overview

Why Three Tiers?

Step 1: Base Role

Step 2: Skill Roles

Trust Policy Template

Permission Policies Per Skill

Self-Assume + Session Tagging (required for all skill roles)

Step 3: Configure in xpander UI

Adding a New Skill

Role Creation Order

Troubleshooting

Security Recommendations

​Architecture Overview

​Why Three Tiers?

​Step 1: Base Role

​Step 2: Skill Roles

​Trust Policy Template

​Permission Policies Per Skill

​Self-Assume + Session Tagging (required for all skill roles)

​Step 3: Configure in xpander UI

​Adding a New Skill

​Role Creation Order

​Troubleshooting

​Security Recommendations

Architecture Overview

Why Three Tiers?

Step 1: Base Role

Step 2: Skill Roles

Trust Policy Template

Permission Policies Per Skill

Self-Assume + Session Tagging (required for all skill roles)

Step 3: Configure in xpander UI

Adding a New Skill

Role Creation Order

Troubleshooting

Security Recommendations