Client.driver_mgr.docker: failed to list pause container

Nomad
1

Hi guys.

I’ve barely begun my adventure - actually am not sure if can call it that yet - with Nomad and on my first client I get this:

    2023-07-29T09:30:36.333+0100 [ERROR] client.driver_mgr.docker: failed to list pause containers for recovery: driver=docker error="Get \"http://unix.sock/containers/json?filters=%7B%22label%22%3A%5B%22com.hashicorp.nomad.alloc_id%22%5D%7D\": dial unix /var/run/docker.sock: connect: no such file or directory"
    2023-07-29T09:30:39.400+0100 [INFO]  client: started client: node_id=4d2daccd-ad48-cce0-a01b-1101332b0c22

I presume this is due to lack of Docker on RHEL derived systems?
If so, I myself prefer to keep it this way - without Docker - therefore an obvious question:

  • is it possible to have Nomad - or any other HashiCorp’s tools for that matter - do its job with podman instead of docker and if yes, then how to disable docker’s bits and enable podman’s?
    Would there be a howto(s) on that?

many thanks, L.

2

Nomad fully supports Podman. Remove docker from the nomad client config. Then check out the Podman docs page on how to set it up. Good luck!

3

Would you know where, which config(s) command my Nomad to use Docker?

My deployment is pretty much vanilla-default from official dnf repos - if devel read this - and I’d expect that those distros(not sure about SUSE) would “default” to podman and have Docker, if not completely “removed” then disabled at least.

4

Default should be /etc/nomad.d/nomad.hcl

Remove the plugin “docker”{} stanza and then add in the Podman stanza outlined in the docs page I posted previously. I’d recommend reading through the Get Started tutorial page.

5

An idiot I’d like think that am I not - I’ve had browsed those already, that is what I usually do - go to man pages first (I failed to find anything specific on disabling drivers/plugins)
When I said ‘vanilla-default’ that was something easy to check - again, devel could easily reproduce this - there is no mention of docker in usual, obvious (to a novice) config/places.

Just to reiterate - log snippet is from a client but server(s) shows the same.
Nomad starts & run - had not chance to put it to work yet - but naturally I don’t know if the error is critical but even it’s not, it’d be good to have Nomad error-free.

thanks, L.

6

Could you post the job file?

7

No jobs yet - as I understand it - this is barely a configured Nomad, only to start, with a config as this:

data_dir = “/opt/nomad/data”
bind_addr = “0.0.0.0”
server {
enabled = false
bootstrap_expect = 1
}
client {
enabled = true
servers = [“10.1.1.99”, “10.1.1.100”, “10.1.1.101”]
}
tls {
http = true
rpc = true
ca_file = “/etc/nomad.d/nomad-agent-ca.pem”
cert_file = “/etc/nomad.d/mine-client-nomad.pem”
key_file = “/etc/nomad.d/mine-client-nomad-key.pem”
verify_server_hostname = false
verify_https_client = false
}

That is all, I start Nomad with systemd and that error shows in the log.

8

Interesting. And just to confirm, Docker isn’t installed on the system at all? What happens if you restart nomad, does the error pop up every time?

9

No, rather disappointing & off-putting than “interesting”.
With long many years of experience with software when I see something like this I head to bugzilla without a delay, but because I’m new completely to Nomad and in hope that devel is active (as in many cases is) here, I decided to consult the forum first, but…
especially not interesting because it must be easily reproducible - get the official repos on a Centos 9s (and perhaps do the minimal config bit as I showed) - and install and start Nomad.

10

Hi @lejeczek,

To keep the logs error-free (the docker related error you reported) and to disable the docker driver on your nomad agents, you can use the denylist option under the client{} block as shown here: client Block - Agent Configuration | Nomad | HashiCorp Developer.

# example:
client {
  options = {
    "driver.denylist" = "docker"
  }
}

And to start using podman driver, follow the Nomad podman driver documentation shared by @shaun.coss above.

I hope this helps!