Docker volume empty

Nomad
1

Binding a path from the alloc/ directory into the /root/.config directory inside of a Docker container results in an empty directory. What am I missing?

    task "app" {
      driver = "docker"
      config {
        image = "custom-image:0.0.1"
        command = "app"
        ports = ["http"]
        volumes = [
          "alloc/root-configs/.config:/root/.config",
        ]
      }
    }

Contents of source directory:

# ls -al /alloc/
drwxrwxrwx 6 nobody nogroup       4096 Oct 29 01:36 .
drwxr-xr-x 1 root   root          4096 Oct 29 01:36 ..
drwxr-xr-x 4 nobody nogroup       4096 Oct 28 23:45 root-configs
drwxrwxrwx 2 nobody nogroup       4096 Oct 29 01:36 data
drwxrwxrwx 2 nobody nogroup       4096 Oct 29 01:36 logs
drwxrwxrwx 2 nobody nogroup       4096 Oct 29 01:36 tmp

# ls -al /alloc/root-configs/
drwxr-xr-x 4 nobody nogroup 4096 Oct 28 23:45 .
drwxrwxrwx 6 nobody nogroup 4096 Oct 29 01:36 ..
drwx------ 4 nobody nogroup 4096 Oct 29 01:36 .config

# ls -al /alloc/root-configs/.config/
drwx------ 4 nobody nogroup 4096 Oct 29 01:36 .
drwxr-xr-x 4 nobody nogroup 4096 Oct 28 23:45 ..
drwxr-xr-x 3 nobody nogroup 4096 Oct 13 23:13 test1
drwxr-xr-x 5 nobody nogroup 4096 Oct 29 01:36 test2

Contents of the destination directory:

# ls -al /root/.config/
drwxr-xr-x 2 root root 4096 Oct 29 01:36 .
drwx------ 1 root root 4096 Oct 29 01:36 ..
2

Hi @SunSparc,

I am not exactly sure what is going on; but could you try interpolating the source directory like ${NOMAD_ALLOC_DIR}/root-configs/.config:/root/.config?

Thanks,
jrasell and the Nomad team

3

Using the interpolation variable:

volumes = [
  "${NOMAD_ALLOC_DIR}/root-configs/.config:/root/.config/",
]

Responds with:

Failed to create container configuration for image "custom-image":
	volumes are not enabled; cannot mount host paths: "/alloc/root-configs/.config:/root/.config/"

I have been assuming that /alloc on the host would be from the root of the host filesystem. Is that not the case?

volumes - (Optional) A list of “host_path:container_path” strings to bind host paths to container paths. Mounting host paths outside of the allocation working directory is prevented by default and limits volumes to directories that exist inside the allocation working directory. You can allow mounting host paths outside of the allocation working directory on individual clients by setting the “docker.volumes.enabled” option to “true” in the client’s configuration. We recommend using mount if you wish to have more control over volume definitions. (source)

When it says bind host paths is that not meaning the Nomad client node host filesystem? Or is that instead meaning the task allocation host paths?

Specifying the source path in either of these ways does not return an error, and they also do not map to anything I need:

volumes = [ ".${NOMAD_ALLOC_DIR}/root-configs/.config:/root/.config/" ]
volumes = [ "./${NOMAD_ALLOC_DIR}/root-configs/.config:/root/.config/" ]

On the Nomad client:

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           393M  836K  392M   1% /run
/dev/vda1        80G   32G   49G  40% /
tmpfs           2.0G     0  2.0G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock

Inside task container:

# df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay          80G   32G   49G  40% /
tmpfs            64M     0   64M   0% /dev
shm              64M     0   64M   0% /dev/shm
/dev/vda1        80G   32G   49G  40% /alloc
tmpfs           1.0M     0  1.0M   0% /secrets
tmpfs           2.0G     0  2.0G   0% /proc/acpi
tmpfs           2.0G     0  2.0G   0% /proc/scsi
tmpfs           2.0G     0  2.0G   0% /sys/firmware

# cat /etc/mtab
/dev/vda1 /root/.config ext4 rw,relatime,discard,errors=remount-ro 0 0
/dev/vda1 /etc/resolv.conf ext4 rw,relatime,discard,errors=remount-ro 0 0
/dev/vda1 /etc/hostname ext4 rw,relatime,discard,errors=remount-ro 0 0
/dev/vda1 /etc/hosts ext4 rw,relatime,discard,errors=remount-ro 0 0

Is the /dev/vda1 reference different inside of the task container?

4

Having same issue.

I want to know what is recommended way to mount alloc/data directory to docker ?

5

Since I could not get this to work, and because the alloc/ directory is already pre-mounted to the Docker container, I decided to adjust my entrypoint command to use this script. It copies the configs I need from the alloc/ directory and then launches the app:

      template {
        destination = "${NOMAD_TASK_DIR}/start.sh"
        perms       = "755"
        data        = <<-EOT
          #!/bin/bash
          set -ex

          cp -r /alloc/configs/.config /root
          /app

        EOT
      }
6

I was able to solve this by using traverse back to alloc from the current task dir and mount.

Ex: “…/alloc/logs:/log”

      driver = "docker"

      config {
        network_mode = "host"
        image = "envoyproxy/envoy:v1.23-latest"
        args = [
          "-c /etc/envoy/envoy.yaml",
          "-l debug",
          "--component-log-level config:info,main:info"
        ]
        volumes = [
          "local/envoy.yaml:/etc/envoy/envoy.yaml",
          "../alloc/logs:/log"
        ]
      }