AWS Secrets Engine - Users tied to root token

We are utilizing the AWS Secrets Engine to create IAM users in AWS. The users were originally created with the root token. We are trying to rotate the root token now, but even when we set the revoke mode to be orphan, the users created with that original root token disappear once revoking. Is there a way to disassociate these users from the token that created them?

Enabled secrets engine and created credentials

vault secrets enable -path=aws-test aws
vault write aws-test/config/root access_key=*** secret_key=*** region=us-east-1
vault write aws-test/config/lease lease=10920h lease_max=10920h
vault secrets tune -max-lease-ttl=10920h -default-lease-ttl=10920h aws-test
vault write aws-test/roles/testme_policy arn=arn:aws:iam::***:policy/testme_policy
vault read aws-test/creds/testme_policy

At this point I can use the AWS console or cli to confirm the user was created with the access key returned and associated with that policy.

Regenerate Root Token and Revoke original

vault operator generate-root -init
vault operator generate-root -nonce=***
vault operator generate-root -decode=*** -otp=***

vault token revoke -mode=orphan <original root token>

At this point I can use the AWS console or cli to confirm the user that was previously created is now gone.

I could be wrong here, but since we are talking here about the AWS secrets engine, and not AWS authentication engine, tokens don’t really apply here. But leases do.

I guess, since you kill the root token, the lease to the AWS secrets engine owned by that token is killed as well. Therefore the AWS users get deleted. This seems the correct intended behaviour, to be honest.

Any particular reason you use the root token to generate those IAM users?

Correct, this is related to the lease going away once the token is revoked.

The reason it was done on the root token actually because of this very reason. We have a process to use AWS Secrets Engine to generate a functional user in AWS that is good for X days. If it was generated via a user with a token-with-ttl, then once that TTL got under 90 days, then the users generated would only be good for whatever time was left on the TTL. Now I realize there were better ways this could have initially been implemented (such as creating a new token for each user good for 90 days) that wouldn’t let us get to this state, but can’t do much about changing the past.

What I’m trying to figure out is if there is a way to
A) Disassociate the lease from the root token (or otherwise transfer)
or
B) Disassociate the user created in AWS from this secrets engine in Vault

Not that I am aware of.

There is nothing in the lease API’s for this: https://www.vaultproject.io/api-docs/system/leases

I assume this is by design. Offering the possibility of transferring leases seems like a dangerous thing to do, with potentially lots of unwanted side-effects.

Does seem like I found a bit of a workaround. Via consul (which is our vault backend) I am able to delete all the keys under vault/sys/expire/id/aws/*. After doing that, revoking the root token does not delete those users (mind you they also won’t auto-expire after the TTL, but at least allows us to get around not being able to revoke the root token and perhaps improving the process for the future).

1 Like

That’s quite a solution :smiley: