Aws_caller_identity - provider profile vs provider environment variable inconsistency

Hi!

I’m wondering whether I’ve discovered a bug or whether this is intended / expected behaviour for aws_caller_identity:

When using aws_caller_identity and two separate aws providers like this:

provider "aws" {
  profile = "primary"
  region  = "eu-west-2"
  version = "~> 2.41"
}

provider "aws" {
  region  = "eu-west-2"
  profile = "secondary"
  alias   = "profile"
  version = "~> 2.41"
}

data "aws_caller_identity" "primary" {}

data "aws_caller_identity" "profile" {
  provider = aws.profile
}

output "primary" {
  value = data.aws_caller_identity.primary.account_id
}

output "profile" {
  value = data.aws_caller_identity.profile.account_id
}

I get the following, as expected:

$ terraform apply
data.aws_caller_identity.primary: Refreshing state...
data.aws_caller_identity.profile: Refreshing state...

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

primary = 75xxxxxxxxx42
profile = 62xxxxxxxxx07

However, if I remove the “profile” from the first provider, and instead supply credentials using AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY like this:

provider "aws" {
  profile = "inshur-infra"
  region  = "eu-west-2"
  version = "~> 2.41"
}

provider "aws" {
  region  = "eu-west-2"
  profile = "inshur-eu-west-2-prod-dns-global0"
  alias   = "profile"
  version = "~> 2.41"
}

data "aws_caller_identity" "primary" {}

data "aws_caller_identity" "profile" {
  provider = aws.profile
}

output "primary" {
  value = data.aws_caller_identity.primary.account_id
}

output "profile" {
  value = data.aws_caller_identity.profile.account_id
}

Then I get the following:

export AWS_ACCESS_KEY_ID=<key_id>
export AWS_SECRET_ACCESS_KEY=<key>
$ terraform apply
data.aws_caller_identity.primary: Refreshing state...
data.aws_caller_identity.profile: Refreshing state...

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

primary = 123123123123
profile = 123123123123

Where 123123123123 is the account ID for the primary aws provider.

I would expect the output from both configs to be the same.

The first config outputs the account_id for each aws profile as I’d like. The second config outputs the account_id for the first provider (which isn’t using a profile in the second example) but for both outputs - this is the part which doesn’t make sense to me.

It should be possible to verify my findings using the two above configs if you have a couple of profiles and a key pair to test with.

If anyone could confirm whether this is a bug, or confirm whether I’m misunderstanding the fundamental usage of aws_caller_identity, I’d appreciate the feedback.

Thanks!

Rob

Hi @roobert,

According to the AWS provider authentication docs, the environment variables – if set – take precedence over your shared credentials file. If you wish to use values from the shared credentials file then you must ensure that the environment variables are not set.

In situations where a configuration needs to work against multiple AWS accounts at once and you don’t want to depend on specific profile names from the shared credentials file, a common alternative approach is to set the environment variables to some credentials for an account that has access to use sts:AssumeRole to act as a set of different roles with different access policies. You can use an assume_role block inside your AWS provider configuration to request that the AWS provider assume another role before taking any actions with that provider configuration.

The Terraform documentation section multi-account AWS architecture describes that model in more detail as part of offering it as a potential solution for systems where each environment belongs to an entirely separate AWS account, but you can use this technique within a single AWS account too if needed.

This shifts the configuration of that access pattern into a combination of AWS IAM and the Terraform configuration, rather than assuming anything about what the user running Terraform might’ve put in their shared credentials file, and can thus make a Terraform configuration more “portable” to run correctly on other systems, or in Terraform Cloud, etc.