Artifact stanza, how does it work?

Nomad
1

Hello!
Let me explain. My service needs some files to start up. These files can be downloaded from a server I set up with an http GET request.
For downloading these files i use multiple artifact stanzas that look very similar to this:

    artifact {
        source      = "http://10.0.0.1/my-file.csv"
        destination = "/data"
     }

And in env variable for a task I pass a path to the file like so:

env {
    DATA_PATH="/data/my-file.csv"
}

The problem happens during a deploy when task is starting it fails with “Container exited with non-zero code”. I check the logs (which are written to a file for convenience) and see that the file does not exists.
So my question is is artifact stanza suppose to download the file before the task starts?
And whether the destination path indicates the path inside the container or the path on a host machine?

2

It is not 100% clearly documented, but since Nomad supports other things besides containers, I would say the path is local to the respective client.

1 Like
3

It’s supposed to download before running the task.

artifact (Artifact: nil) - Defines an artifact to download before running the task. This may be specified multiple times to download multiple artifacts

1 Like
4

Thank you for your replies!
Do you by any chance know what happens when I provide an absolute path in destination?

5

Can’t believe i missed this part, read this page 10 times at least.

6

Sometimes you can’t see the forest for the trees. :wink:

7

I don’t know what will happen with an absolute path. Have to try it for myself.

8

Ok, I was going to try it too, but figured I ask first in case you have some insights. Thank you for the help!

9

So you’ll try and give us your results. :slight_smile:

10

Got to it at last. The result is something you might expect: when I use absolute path as in /data/ nomad creates the folder in the allocation root directory on the same level with local/, secrets/ and tmp/.
Thank you for your help!

1 Like
11

Hi @rebrendov,

Glad to hear you figured it out.

This is also documented. Files downloaded by the artifact stanza will be saved in a path “…relative to the root of the task’s directory” (see atifact's destination parameter).

Downloaded files are easily accessible by your instance if stored in one of the instance’s Task directories. These directories are then made available to the allocation in various ways. “The path to the directories can be read through the NOMAD_ALLOC_DIR , NOMAD_TASK_DIR , and NOMAD_SECRETS_DIR environment variables.”

I’m not 100% sure whether the /data directory will be automatically mounted, so this is something you should test.

2 Likes
12

Hey @blake,

Yeah I’ve read that one. And that’s partially what confused me. Because in the destination stanza documentation the default value is shown as a relative path (i.e. local/). I probably should’ve payed closer attention to the “…relative to the root of task’s directory” you mentioned, but it wasn’t obvious to me that it applies to absolute path as well.

Another source of confusion for me was the narrowness of my understanding of what task is in Nomad. Because so far I’ve only used Nomad for container orchestration when I think "Nomad’s task" I think Docker's container which is wrong.

The last sentence in your comment describes exactly what I was expecting. But alas it’s not the case. Tested this today and with absolute path, and it seems that by default Nomad creates only the following mounts in the docker container:

"Mounts": [
            {
                "Type": "bind",
                "Source": "/tmp/NomadClient515032879/e954725b-e4ae-5c88-72ac-5fd45f44e654/alloc",
                "Destination": "/alloc",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/tmp/NomadClient515032879/e954725b-e4ae-5c88-72ac-5fd45f44e654/redis/local",
                "Destination": "/local",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/tmp/NomadClient515032879/e954725b-e4ae-5c88-72ac-5fd45f44e654/redis/secrets",
                "Destination": "/secrets",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "volume",
                "Name": "1693c07d2a47d4dd5075e71eeed6382fb49a631b95062c23d16078879e49e183",
                "Source": "/var/lib/docker/volumes/1693c07d2a47d4dd5075e71eeed6382fb49a631b95062c23d16078879e49e183/_data",
                "Destination": "/data",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ]

Well, the more you know.
I guess it’s easier to just use default instead of setting your own path in artifact's stanza destination when you’re using containers.

1 Like