Changing Vault path between allocations

Nomad
1

New Nomad user here :slight_smile:

I’m trying to run multiple allocations of a single job but have a different Vault path between the jobs (I want each allocation to access a different secret). Any ideas on how to do this / best practices?

I need to use allocations so I can also use the spread stanza.

Thanks!

2

Hi @achen4290

Thanks for using Nomad!

Is the Value the same for all allocations on a given client? If so, maybe you could use an environment variable?

3

Hi Derek,

Unfortunately the secret values and Vault paths are different for every allocation, regardless of client (which is why I’m in this predicament :sweat_smile: ) The current solution I have in mind is to have a prestart lifecycle task for every allocation that reaches out to another server I deploy to get the path to the secret for that specific allocation. Do you have any better ideas / ideas on how to improve this one?

Cheers,
Andrew

4

Hi @achen4290, what actually determines the association between an allocation and the path it should be using? Or are you saying there are N allocations and N paths, and each allocation should be assigned a unique path?

5

Hi Seth, the latter is what I’m looking for (your description is great :slight_smile:). Just N allocations and N paths, I don’t care which allocation gets which path but I’m wondering if there’s a way to, given N paths, divide the paths among N allocations so there’s a 1:1 mapping. Thanks!

6

In that case you might be able to get away with the NOMAD_ALLOC_INDEX runtime environment variable Nomad sets on your behalf. There’s some caveats to be aware of so make sure you read the docs:

At the other extreme you could use Consul session locking to do something like have each instance of your task try to acquire a session associated with a unique path until it finds one not yet taken. If you go this route we have an example (still in PR) nomad pack for Consul session locking.

And surely there are other solutions, it’s distributed computing after all :thinking:

7

I think NOMAD_ALLOC_INDEX is perfect for what I’m trying, can’t believe I missed it in the documentation :sweat_smile:. Thanks for the help!