Parameterized (META_) Interpolation Scope

Nomad
1

I was wondering, why the interpolation of parameterized jobs (with the META_ variables) does not apply to the “user” key inside the “task” stanza.

For example, the interpolation of HCL variables works just fine for the key “user” inside the “task” stanza:

    task "test-task" {
      driver = "exec"
      user = var.uservar

However, this does not work when I parameterize the job and use the META_ variable:

    task "test-task" {
      driver = "exec"
      # Fail: unable to find user ${NOMAD_META_USERMETA}
      #user = NOMAD_META_USERMETA
      #user = "${NOMAD_META_USERMETA}"

Please find a complete example that was used with nomad agent -dev below.

variable "uservar" {
  type        = string
  description = "Unix user to launch the job with"
}

job "test" {
  datacenters = ["dc1"]
  type = "batch"

  parameterized  {
    payload       = "forbidden"
    meta_required = ["usermeta"]
  }

  group "test-group" {
    count = 1

    task "test-task" {
      driver = "exec"
      # Fail:
      #user = NOMAD_META_USERMETA
      #user = "${NOMAD_META_USERMETA}"
      #
      # Pass:
      user = var.uservar

      config {
        command = "/bin/sh"
        args = ["-c", "sleep 3600"]
      }
    }
  }
}

When run using the parameterized variable NOMAD_META_USERMETA for the “user” key, the job produces following output:

    2022-02-15T07:01:51.998+0100 [ERROR] client.alloc_runner.task_runner: running driver failed: alloc_id=88b522c7-48b4-eec8-51e5-74b72db8b8fe task=test-task error="failed to launch command with executor: rpc error: code = Unknown desc = container_linux.go:367: starting container process caused: setup user: unable to find user ${NOMAD_META_USERMETA}: no matching entries in passwd file"

This output is clear to me. The variable did not get interpolated, rather, it was interpreted as raw text string and the user could not be found.

What is not clear to me, however, is where exactly the interpolation of the different variables (either HCL variables or variables from the parameterized job, META_) can be used (i.e., for which stanzas and keys the interpolation of these variables applies).

I could find some details on the valid scope for interpolated variables here:

However, these docs don’t elaborate on the scope for the parameterized variables (META_) inside the “task” stanza in general. It seems like only the scopes “env” and “constraint” are treated there. Also, from the many examples and my experience it was clear to me that META_ variables are indeed interpreted, when used inside the “config” section of the task.

There is one interesting comment there, however:

runtime environment variables are only defined after the task is placed on a node.

I can see that it makes no sense to evaluate “constraints” after the task was already placed on a node.

But does this also apply to the “user” key in the “task” stanza? This could be parameterized beforehand, independent of the placement of the particular task, couldn’t it?

In general, I might also still be a bit confused about when to user HCL input variables (Input Variables - HCL Configuration Language | Nomad by HashiCorp) compared to parameterized jobs (parameterized Stanza - Job Specification | Nomad by HashiCorp).

Thanks for any hints.

2

Very interested in this same topic. I actually want to parameterize the hostname and use that in the constraint for attr.unique.hostname… I know runtime variables are interpolated after placement, but I would like this meta var to be evaluated prior to placement.

I would have to have one parameterized job per host