Docker container has permission issues when deployed through Nomad, but not when deployed through Docker CLI on same host

Nomad
1

Pretty confused here, must be missing something obvious.

Trying to deploy Nextcloud on my cluster, without persistent storage for now, even.

Here’s my jobspec:

job "nextcloud" {
  region = "global"
  datacenters = ["dc1"]
  namespace   = "default"
  type        = "service"
  
  group "nextcloud" {
    network {
      mode = "bridge"
      port "http" {
        to = 80
      }
      port "db" {
        to = 5432
      }
    }

    task "nextcloud" {
      driver = "docker"

      config {
        image = "lscr.io/linuxserver/nextcloud:latest"
      }

      resources {
        cpu    = 2000
        memory = 4048
      }

      env {
        NEXTCLOUD_TRUSTED_DOMAINS = "[redacted]"
        TRUSTED_PROXIES = "192.168.1.216"
        TZ = "Europe/Berlin"
        PGID = "1000"
        PUID = "1000"
      }

      service {
        name = "nextcloud"
        port = "http"

        tags = [
          "traefik.enable=true",
          "traefik.http.routers.nextcloud.rule=Host(`[redacted]`)",
          "traefik.http.routers.nextcloud.tls=true",
          "traefik.http.routers.nextcloud.tls.certresolver=myresolver",
        ]
      }
    }
  }
}

Immediately after deploying through nomad, it fails with:

chown: changing ownership of '/app': Operation not permitted
chown: changing ownership of '/config': Operation not permitted
chown: changing ownership of '/defaults': Operation not permitted
mkdir: cannot create directory ‘/var/lib/nginx’: Permission denied
s6-rc: warning: unable to start service init-folders: command exited 1
chown: changing ownership of '/etc/crontabs/abc': Operation not permitted
crontab: setegid: Operation not permitted

… which is quite confusing to me, because all those folders are obviously within the container. Why are there permission issues?

Even when I change the container’s PGID and PUID env vars (which affect the user the process within the container runs as) to 0:0, I get another permission error:

mkdir: cannot create directory ‘/var/lib/nginx’: Permission denied
s6-rc: warning: unable to start service init-folders: command exited 1

… which is even more confusing to me.

And here’s the thing: When I start it using the Docker CLI on the same host, with the same config, like this:

docker run -d \
  --name=nextcloud \
  -e PUID=1000 \
  -e PGID=1000 \
  -e TZ=Etc/UTC \
  -p 443:443 \
  --restart unless-stopped \
  lscr.io/linuxserver/nextcloud:latest

… everything works fine! So, same host, same config, same Docker daemon, same image… but it doesn’t work through Nomad. Docker / the container itself is running as root in both cases too.

What could this be? I must really be missing something obvious here.

2

What user is the Noamd agent running as?

Can you force the user to root for the docker driver to see if the error goes away?

just-an-idea