TF unable to assume role

Terraform
1

Hi guys!

There is something going on here with TF and AWS.

I have an AWS IAM role (terraform-backend-role) deployed to my account that gives TF access to the DynamoDB table and the S3 Bucket.

I can successfully assume the role locally on my laptop:

ASSUMEDROLEUSER arn:aws:sts::XXXXXX:assumed-role/terraform-backend-role/AWSCLI-Session-master     AROAROCV3B5YDTAX2HBJB:AWSCLI-Session-master
CREDENTIALS     ASIAROCV3B5YDV5NCBPK    2022-12-05T21:54:51+00:00       UA___A==

However, when running terraform init I get the following error:

❯ terraform init --backend-config="role_arn=arn:aws:iam::XXXXX:role/terraform-backend-role"

Initializing the backend...
â•·
│ Error: error configuring S3 Backend: IAM Role (arn:aws:iam::XXXXX:role/terraform-backend-role) cannot be assumed.
│ 
│ There are a number of possible causes of this - the most common are:
│   * The credentials used in order to assume the role are invalid
│   * The credentials do not have appropriate permission to assume the role
│   * The role ARN is not valid
│ 
│ Error: NoCredentialProviders: no valid providers in chain. Deprecated.
│   For verbose messaging see aws.Config.CredentialsChainVerboseErrors

Terraform version:

❯ terraform version
Terraform v1.3.4
on darwin_arm64

I don’t know what I’m missing here.

2

Hi @manu.nz!

Unfortunately with the information you shared I don’t really have anything to add beyond what the error message already mentions. :confounded:

I think the fact that you can assume this same role using your own credentials on your laptop means that you can rule out “The role ARN is not valid” as a possible explanation, but that still leaves the other two in question:

  1. The credentials used in order to assume the role are invalid
  2. The credentials do not have appropriate permission to assume the role

The first of these is talking about whether the direct credentials in your execution environment are valid. For example, this might be the credentials in your AWS_ACCESS_KEY_ID and similar environment variables, or credentials in your ~/.aws/credentials file, or similar. This error could appear if those credentials are missing, invalid, or expired.

The second of these is discussing another situation where your direct credentials are valid but they authenticate as a user or role that isn’t permitted to assume the role. The “assume role policy” of the role is what decides whether your credentials have appropriate permission to assume the role.

I would suggest investigating both of those possibilities in more detail to see if you can determine which of those two possibilities is the cause.

3

Hi @apparentlymart , sorry for the lack of information!

The way I’m authenticating locally is via aws configure sso. Which means that Terraform should pick up those credentials as well to authenticate.

in fact, when I run aws sts get-caller-identity I get a successful response:

XXXX    arn:aws:sts::XXXX:assumed-role/AWSReservedSSO_AWSAdministratorAccess_16d6a284d928080c/useremail@example.com  AROAVWF7AMKR23APCW7ZK:useremail@example.com

My backend configuration looks like this:

terraform {
  backend "s3" {
    bucket         = "example-terraform-backend-tfstate"
    key            = "example/terraform-example.tfstate"
    region         = "ap-southeast-2"
    dynamodb_table = "terraform-state-lock"
    encrypt        = "true"
    role_arn       = ""
  }
}

You see the role_arn is empty. That’s why I use the following command to initialize TF: terraform init --backend-config="role_arn=arn:aws:iam::XXXX:role/terraform-backend-role"

Edit:

I can also see in my env that the AWS_PROFILE variable is set: AWS_PROFILE=ssoprofile

So, TF should pick that up.

4

OK - I had other credentials in the ~/.aws/credentials file which had the same name as the profile I’m using. That’s why it was failing.

Thanks for the help @apparentlymart !

5

Thanks for confirming!

In case someone finds this topic in future and isn’t sure, just want to be explicit that aws sts get-caller-identity is a good way to see whether the AWS CLI itself has valid credentials available. Unless you’ve overridden the credentials in Terraform to be different than the ones AWS CLI uses, that’s a reasonable way to eliminate possibility 1 from my previous comment and thus focus on possibility 2.

The success of aws sts get-caller-identity means that there are some valid credentials configured, but it doesn’t say anything about whether those credentials represent a principal that has access to assume the role.

Although it wasn’t important for @manu.nz to diagnose the problem here, my next step for debugging after eliminating possibility 1 would be to try to assume role with the CLI too:

aws sts assume-role --role-arn "arn:aws:iam::XXXX:role/terraform-backend-role"

I would expect this to return an error similar to the one Terraform returned, thereby confirming that the auth principal returned by aws sts get-caller-identity doesn’t have permission to assume this role. That means either that different credentials for a different principal are needed, or that the “assume role policy” for this role needs to be modified to include that principal.

If you have a similar problem in future and find this and the above isn’t sufficient to help you diagnose the problem, please feel free to start a new topic and then include the results from running these diagnosis steps in your opening message.

1 Like