Where does the artifact stanza runs and under what user?

Nomad
1

Hi all,

I am trying to figure out how not to store AWS credentials in my nomad hcl file / job that is spinning up a docker container that requires an artifact from S3.

In theory this should work, if I have the default profile in place:

  artifact {
    source = "mybucket.s3-eu-west-2.amazonaws.com/myfolder/myotherfolder/myfile.tar.gz?archive=false&region=eu-west-2&aws_profile=default"
    mode = "file"
  }

I have configured the .aws/config and .aws.credentials files on the container. First in the standard user’s home dir. Later, I tried in the /root, in /local and nothing seems to work. I have also tried it on the host’s /root, since the nomad client is running as root. No joy.

Every time I am trying to run the job, I get the generic message from go-getter that pops up if your aws config is not in place:

failed to download artifact “mybucket.s3-eu-west-2.amazonaws.com/myfolder/myotherfolder/myfile.tar.gz?archive=false&region=eu-west-2&aws_profile=default”: MissingRegion: could not find region configuration

I have also tried to use the “options” fir the profile and region of the artifact stanza with the same results.

If I set up the .aws folder/files on the host of the nomad client and run go-getter directly without nomad in the /root it does work properly with:

./go-getter “mybucket.s3-eu-west-2.amazonaws.com/myfolder/myotherfolder/myfile.tar.gz?archive=false&region=eu-west-2&aws_profile=default” “myfile.tar”

If I stop using the profile and just use the key and region explicitly in the hcl file all is working fine, but that is something I really would like to avoid.

Can someone shed some light on what am I doing wrong?

Many thanks,
Peter

2

Hi @Peter :wave:

Which version of Nomad are you running?

I wonder if your problem is related to Cannot get S3 private bucket artifact to download · Issue #8454 · hashicorp/nomad · GitHub which was fixed on Nomad 1.0.2.

3

Ah, apologies, I’ve forget to respond since I have put this part of the project aside for a bit.

I am running the latest nomad (1.1.1).

I have set up a dev environment and planning to add a bit of extra logging to go-getter and see what is going on (without a better idea)