How to assume role to read KMS keys?

Boundary
1

Terraform allows assumption of roles with the AWS provider. However it seems the Boundary provider doesn’t use the AWS provider, nor provide an option for setting the profile to assume to read the key?

The only way I can get this to work is to login and set the AWS_PROFILE environment variable before running Terraform, which defeats CI testing implementations.

Would be great if it supported assuming profiles similar to or making use of Terraform’s AWS provider.

kms "awskms" {
  purpose    = "root"
  region     = "us-east-1"
  assume_role {
    role_arn = "arn:aws:iam::0000000000:role/iam-identity-foobar"
  }
}
2

Hey there,

From where are you running Boundary? Is this on an EC2 instance within AWS or somewhere else? Under the authentication section for KMS AWS, we point out that:

AWS authentication values:

Note: The client uses the official AWS SDK and will use the specified credentials, environment >credentials, shared file credentials, or IAM role/ECS task credentials in that order, if the above AWS specific values are not provided.

If you’re on an EC2 instance, you can leverage the role as an instance profile. Under the hood, we’re running the AWS SDK, and so all env vars and other authentication mechanisms should happen automatically.

3

When using the Terraform provider you are running from a workstation or CI system, not from the instance node.

Yes, it would be possible to create a CI node in EC2 which has a god-like instance profile, but this is a lot less secure than specific role assumption rights given to specific jobs.

If you are using the AWS provider with role assumption, you can’t use the environment variables for all the reasons this ability was added to the AWS provider. (multiple roles are used for this deployment)

The AWS provider is based on the same SDK, so it would have the same abilities if you added the same attributes to the Boundary provider schema.

4

Since it doesn’t seem I overlooked anything, I’ve created Boundary provider needs assume_role · Issue #62 · hashicorp/terraform-provider-boundary · GitHub to track this

5

I’m still not following the workflow clearly here. In the example, you have a awskms block, that block specifies a KMS key, not IAM configuration, for encrypting different types of Boundary traffic. How are you using Boundary from CI? Can you clarify if this is something you’re doing from Terraform (and the specific provider, it sounds like you’re using Boundary and AWS providers), or Boundary itself?

Thanks!

6

Ah, the ticket had the info I was looking for, going to take this conversation there - thanks for doing that.

7

Yes. In order to read that key, one must assume the correct IAM profile. There’s no way to do that today except by setting environment variables… which breaks all other AWS operations in the same Terraform run.

I’m asking you to add the ability to assume an AWS profile in order to read the KMS provided.

8

@malnick it seems like you’ve added this but the docs aren’t clear on how to use it. I can read the kms key from the instance profile using aws command line. For example, this config:

$ cat ~/.aws/config
[profile security]
region = us-east-1
role_arn = arn:aws:iam::1234567890:role/boundary-worker
credential_source = Ec2InstanceMetadata

$ AWS_PROFILE=security aws kms describe-key --region=us-east-1 --key-id=arn:aws:kms:us-east-1:1234567890:key/0123abc-ab12-3456-cd78-ef90123456
{
    "KeyMetadata": {
        "AWSAccountId": "1234567890",
...

How do I represent this in the awskms config? I recognize that I could template out the aws config and set the profile but that’s a lot of touchpoints. It seems to me that this should work? AWS KMS - KMSs - Configuration | Boundary by HashiCorp

It doesn’t, however. Your docs indicate that this parameter is also required, but there’s not a single word of what this value should be? AWS KMS - KMSs - Configuration | Boundary by HashiCorp

:question:

I tried creating a file that only boundary user could read or write to and specifying it, thinking perhaps it wanted storage location… but it doesn’t solve it.

Perhaps most frustrating is that even with trace logging there’s nothing logged about what is happening here:

Mar  1 23:11:39 localhost boundary[3209046]: Error parsing KMS configuration: error fetching AWS KMS wrapping key information: AccessDeniedException:
Mar  1 23:11:39 localhost boundary[3209046]: #011status code: 400, request id: 179defbd-4dfb-48f2-aec5-dcec7db78ec3
Mar  1 23:11:39 localhost systemd[1]: boundary-worker.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
Mar  1 23:11:39 localhost systemd[1]: boundary-worker.service: Failed with result 'exit-code'.
9

FYI I have tried both of these configurations, and neither one works. Both output the exact same error messages even at trace level so it’s hard to know if the values are even being read

kms "awskms" {
  purpose    = "worker-auth"
  region     = "us-east-1"
  kms_key_id = "arn:aws:kms:us-east-1:1234567890:key/_SNIP_"
  role_arn   = "arn:aws:iam::1234567890:role/boundary-worker-staging"
  web_identity_token_file = "/etc/boundary.d/token"
}

Then I tried using the aws config that works fine for the aws cli as shown in my previous message, along with this worker config:

kms "awskms" {
  purpose    = "worker-auth"
  region     = "us-east-1"
  kms_key_id = "arn:aws:kms:us-east-1:1234567890:key/_SNIP_"
  shared_creds_filename = "/etc/boundary.d/aws_config"
  shared_creds_profile = "security"
}

Still no luck.

10

I landed on the originating Github issue trying to work through a similar problem. We use cross-account assume roles heavily in our CD environment, which is something the AWS provider handles well (including using provider aliases to be able to access multiple accounts’ resources from one module).

Digging through the code, it looks like the problem might be way down in awsutil.GenerateCredentialChain. The code there seems to assume the only AssumeRole method will be via IAM’s OIDC integration since the only call is via NewWebIdentityRoleProvider. (@jorhett , this is what the web_identity_token_file feeds into - the WebIdentityToken parameter of the underlying AWS API call, not what we need).

It looks like the solution is being able to add an AssumeRoleProvider to the credentials chain in that method with the role_arn from the KMS config HCL, but we’ve reached the limit of how much I understand Go (including how to hack on this library to test locally).

1 Like
11

I’ve created a PR Add roleprovider for role assumption by ec2 instance by jorhett · Pull Request #33 · hashicorp/go-secure-stdlib · GitHub

This isn’t acceptable, it doesn’t contain tests, etc. I don’t have time to fight through the hashicorp tests and standards, etc. I’m really hoping @jeff will take this and run with it

1 Like
12

@malnick 14 months ago you told me:

If you’re on an EC2 instance, you can leverage the role as an instance profile.

I realize this project is pre-alpha, whatever. And I’m not expecting 24 hour turnaround… but 14 months later this still isn’t possible. You’ve added Saml-based web token role assumption (which is a child class of this) without handling the base scenario.

This remains the blocker for rollout/usage for us, and quite a few others given the comments. I’ve gone far enough to write a PR to fix it. Can we get some attention on this please?

13

16 months later and not a single reply to either this thread or the PR I provided.

Apparently neither @jeff nor @malnick care.

14

Hi @jorhett! Apologies for the delay. We will be looking at the PR you’ve filed; since it is outside the “scope” of a single product, we’ll be working on this as a cross-team effort. I’ll do my best to shepherd this along. Thanks for your patience and your initial efforts to get this going! It is truly appreciated.

1 Like
15

Any update on this? Would love to see this land.

1 Like
16

We are still working on it! I’ll check in on it again and try to continue to nudge it along.

17

This is being picked up in our next sprint. I can’t make any guarantees it’ll be finished in that sprint, but I know it’s being actively worked on - and thanks again for your patience!

1 Like