AD Service Account Library Checkout & Password Retrieval

Vault
1

I would like to be able to use the Service Account Library feature of the AD engine, but I’m running into challenges with my use case.

In my production deployment, several groups of microservices need to run as a common or consistent identity. Each group can have a different identity but the microservices within the group should be the same identity.

Most services get service account credentials from Vault at deploy-time/startup. Each service may be redeployed or restarted independently of other services.

Using the AD engine without the Library functionality, I can configure the services to call the GET /ad/creds/:role_name endpoint with a static role_name to get the current AD password for that service account. However, if this call results in password rotation because the TTL has passed, only the newest microservice has the correct password. All others will begin failing until they are restarted or redeployed.

In Vault versions before 1.3.0, I explored using a blue/green service account scheme, where there are always two or more active service accounts with overlapping TTL windows identical permissions and attributes. This works, but requires an orchestrator to know the names of all the service accounts in the scheme as well as tracking the “current” or “active” service account. And it sure seems ripe for migration to the new check-out functionality!

I can create a Vault AD Library that contains enough “clone” service accounts (meaning they differ only in name) that any microservice in the group can check out a service account independently. This is difficult to scale and creates a risk of failure as soon as the n+1th service tries to check out an account. So that option is out.

I can use an orchestrator to check out a service account from the Vault AD Library on a predefined schedule, letting the previous check-out expire on its own. The orchestrator then records or communicates the name of the checked-out account as “current” for any services that spin This is similar to the blue/green scheme in that it requires the orchestrator to track the “current” account, but removes the requirement for the orchestrator know the names of the service accounts. It also allows services with the previous account name and password to keep running with until they get the updated credentials.

Here’s what I’m running into, though.

  • While the simple GET /ad/creds/:role_name endpoint will always return the current password, only changing when the TTL has passed, there does not seem to be a way to retrieve the password of a checked-out account after checkout.
  • The password is only returned once: in the initial call to POST /ad/library/:set_name/check-out
  • ^ This means the orchestrator must store and manage the password securely throughout its lifetime. (Certainly it can be stored as K/V in Vault, but adding code to the orchestrator to manage the password creates a new point of risk.)
  • If a service passes the same service account name to GET /ad/creds/:role_name later, Vault doesn’t seem to know that it’s already managing the credential and immediately rotates the password.

What I’d love to see:

  • Enable a borrower or delegate to call an endpoint in /ad/library and pass in a service account name (and checkout token?) to retrieve the account’s current (non-rotated) password. Throw an error if GET /ad/creds/:role_name is called with the same role_name as a checked-out service account.

or

Thank you!

I’m also open to other great design ideas and I’m happy to share other options I’ve explored.