Podman + userns, logging, and podman-specific configuration

Nomad
1

Hi all,

I am trying to evaluate Nomad for my use case - so far it seems to be pretty close to what I need. I have a test cluster with three nodes using Fedora 35 + podman + btrfs + crun + nomad-driver-podman up and running. I am able to submit a job to the Nomad cluster, see its ports, stop it, scale it, and so on. I have the following questions:

  1. How can I make Nomad run containers in an unprivileged user namespace, as in podman --userns=auto? So far the top-level process of the docker.io/library/redis:latest container runs as systemd-oom (UID 999), as seen with ps axu from the host OS. When I run podman run --userns=auto -it docker.io/library/redis:latest manually, the UID of the toplevel process respects the UID range from /etc/subuid.

  2. Does logging to Nomad work with podman driver? When I run nomad alloc logs d2f3035a, I get an empty output, even though the allocation ID seems to be valid.

  3. I have problem adding podman-specific configuration to my nomad.hcl and the job description files. I use the documentation from here: Drivers: podman | Nomad by HashiCorp - when I for example add the top-level section plugin "nomad-driver-podman" { } to the nomad.hcl config, as described in the first example of the “Plugin Options” section in that page, nomad fails with:

nomad[30862]: ==> Error starting agent: failed to create plugin loader: failed to initialize plugin loader: parsing plugin configurations failed: 1 error occurred:
nomad[30862]:         * plugin "podman" (driver): failed to parse config:  Invalid label: No argument or block type is named "disable_log_collection".
nomad[30862]:     2022-03-18T13:25:50.450+0100 [ERROR] agent: error starting agent:
nomad[30862]:   error=
nomad[30862]:   | failed to create plugin loader: failed to initialize plugin loader: parsing plugin configurations failed: 1 error occurred:
nomad[30862]:   |         * plugin "podman" (driver): failed to parse config:  Invalid label: No argument or block type is named "disable_log_collection".
nomad[30862]:   |

The similar thing happens when I add logging = { driver = "nomad" } section inside the task config { } section. When trying to run the job, I get the following error message:

Invalid label: No argument or block type is named "logging".

Thanks in advance for any hints.

-Yenya

2

I just came across this - you are attempting to use unreleased features. Building GitHub - hashicorp/nomad-driver-podman: A nomad taskdriver for podman containers from main gave the logging features you are referring to.