Task API api.sock availability

Nomad
1

Hi all, I’m running nomad in dev mode like this:

“nomad”, “agent”, “-dev”, “-config=/etc/nomad/client.conf”, “-bind”, “0.0.0.0”

with client.conf not containing much.

When I open a shell on a task, I cannot see any $NOMAD_SECRET_DIR/api.sock .

I was wondering if it could be a sock file name length issue, but I can’t see any warning in the log that the api.sock haven’t been created.

My secret dir gets filesystem mounted by my nomad (crasher being the name of my task)

-/secrets /dev/sdc[/tmp/NomadClient2884966027/9a73c312-32cf-4100-2700-dde90c222bf7/crasher/secrets]

I’ve also tried enabling ACL with no success having this api.sock being generated.

Wondering what I am doing wrong?

2

Hi @jeteve,

Could you confirm what version of Nomad you are running and what operating system you are using? Do you have the logs handy also from when the agent starts the task?

Thanks,
jrasell and the Nomad team

3

Hi! I found what the problem is.

I run nomad in a container as part of a devcontainer in vscode.

So far so good. Works well and the other Devs don’t have to think about how to install it and run it.

When nomad builds a task directories, it mounts a tmpfs as it’s running as root within the container.

I also need this as the docker driver requires root.

The trouble is that a tmpfs cannot be shared between containers. So the tmpfs that’s mounted by the nomad container is not mapped to the secret dir in the task container.

The task container gets a plain filesystem path based secret bind mount, which is empty. No .mount, no api.sock in this one.

So it’s pretty much checkmate in this configuration as root nomad triggers tmpfs.

I guess a good way to solve it would be to make it a config (tmpfs switch off for task local dirs), or simply avoid tmpfs in dev mode.

Cheers,

Jerome.