Workload identity and custom vault policy

Nomad
1

Hi.
Just reading the new workload identify features in Nomad 1.7. While it seems interesting, it also seems far more complicated to manage than the old way. So it’s a bit sad to see it deprecated and planned to be removed in Nomad 1.9 :frowning:

The doc about how the workload identity is still a bit unclear. We can define templated vault policy all workload will inherit, like

path "secret/data/{{identity.entity.aliases.auth_jwt_3a9350fe.metadata.nomad_namespace}}/{{identity.entity.aliases.auth_jwt_3a9350fe.metadata.nomad_job_id}}/*" {
  capabilities = ["read"]
}

(BTW, this is already a pain to manage automatically as we have to lookup the accessor ID of the JWT auth before writing the policy template). But, how to grant additional, custom vault policy to a specific task in a job ?

2

But, how to grant additional, custom vault policy to a specific task in a job ?

Hi @dbd! The way to solve this is to use a different vault.role for the jobs that need to use a different set of policies. Otherwise it’ll use the default_role from the auth method.

3

So, for every job, I’d now need to create my custom vault policy (as before), but also a new token role to map this policy ? This looks like a huge usability regression (or maybe I misunderstood)

4

@dbd thanks for the quick feedback.

We’re broadly committed to this direction to reduce the overhead and security risk around manual/static token management, but we do want to ensure that by the time people have to adopt this its a better UX.

Two things that we could use some feedback on (from anybody reading this not just @dbd )

  • How far can you get with Templated Policies? If you can have a single role attached to one of these (or a few pre-created roles), then ideally you don’t even have to go and create new policies for new jobs. - I realize this probably forces you to impose a more rigid structure on secrets paths. Is that a deal breaker?
  • How painful is role creation when you can’t use an existing templated policy/role? It sounds like this answer is “pretty painful” in your case?

We don’t have this functionality yet (and is a speculative idea at this point), but if we provided a way to pass thru identity data to roles, and then the roles could “bind” to specific policies based on that, could that work? So some role grants access to any policy named “nomad--” or something like that.

5

While templated policies could work for I’d say ~50% of my workloads (I’m already using a structured hierarchy for my KV secrets, I’ll probably have to move a few things around, but still doable). But for the other ~50%, I need specific vault policies (issue certs on a specific PKI, get a consul or nomad token, read secrets from another workload etc.). For those, it’s too specific to be able to rely on a few templated policices. I currently automate the creation of all those custom policices (well, I write it manually, only the vault write is automated), and can easily add this to a task with

vault {
  policices = ["mycustompolicy"]
}

As Nomad servers are able to issue tokens with any policy attached (except a few high privileged which I have blacklisted), there’s nothing more to do.

I can probably work on automating the role creation, but it’s one more step, one more indirection (harder to understand and debug), without bringing any advantage (as I will have to create one role for each of those custom policices). It just seems less practical than the old way.

Indeed, if there was a way to somehow pass the policy I want as a metadata attached to the identity, so that the role could map specific policies to the final token, it’d be helpful (in fact, that’d emulate the old way of managing policies :wink: )