Nomad driver, dynamic args

Nomad
1

I am trying to create a dynamic list of args for a nomad driver (in this case “exec”).

As a proof of concept, I tried following task. This works perfectly fine and the service (minio) starts up correctly (I have 3 different jobs each running minio on different nodes).

task "minio" {
      driver = "exec"

      resources {
        cpu    = 500 # MHz
        memory = 768 # MB
      }

      volume_mount {
        volume      = "minio-storage"
        destination = "/minio-data"
        read_only   = false
      }

      config {
        command = "minio"

        args = concat([
            "server",
            "--address", ":9001",
            "--console-address", ":9091"
          ],
          [
            "http://10.0.1.3:9001/minio-data",
            "http://10.0.1.5:9001/minio-data",
            "http://10.0.1.6:9001/minio-data"
          ]
        )
      }

In the next step I tried to do this dynamically by using the internal nomad service discovery. I added a template block and adjusted the args block:

      template {
        destination = "$${NOMAD_TASK_DIR}/env.vars"
        env         = true
        change_mode = "restart"
        data        = <<EOF
          MINIO_HOSTS=
          {{- range nomadService "minio-api" -}}
          http://{{ .Address }}:{{ .Port }}/minio-data {{ end }}
        EOF
      }

      config {
        command = "minio"

        args = concat([
          "server",
          "--address", ":9001",
          "--console-address", ":9091"
          ],
          split(" ", trim(env["MINIO_HOSTS"], " "))
        )

This results in a different behaviour (minio complains on startup). I replaced command with echo to see if there is any difference and to make sure MINIO_HOSTS is correctly populated.

In both cases I get exactly the same output:

server --address :9001 --console-address :9091 http://10.0.1.3:9001/minio-data http://10.0.1.5:9001/minio-data http://10.0.1.6:9001/minio-data

Somehow the arguments passed to minio are different. I haven’t been able to figure out why yet =/

2

Looking at the driver definition part of the job via the nomad UI I get:

          "Name": "minio",
          "Driver": "exec",
          "User": "",
          "Config": {
            "command": "minio",
            "args": [
              "server",
              "--address",
              ":9001",
              "--console-address",
              ":9091",
              "${env[\"MINIO_HOSTS\"]}"
            ]
          },

So that means concat worked correctly but somehow the parser removed split and trim?

3

I just ended up creating a bash script with the template block, which is totally fine in my book.

      template {
        destination = "$${NOMAD_TASK_DIR}/minio.sh"
        change_mode = "restart"
                 
        data        = <<EOF
          minio server \       
          {{ range nomadService "minio-api" -}}http://{{ .Address }}:{{ .Port }}/minio-data {{ end }} \                
          --address :9001 --console-address :9091
        EOF
      }    

      config { 
        command = "bash"        
        args    = ["$${NOMAD_TASK_DIR}/minio.sh"] 
      }