Issues with migrate Nomad Vault token to JWT Workloads

Nomad
1

hi,

I try to follow the example for switching Nomad from Token to Workloads and JWT, when receiving secrets. I have couple of issues .. and no idea, what is missing.

  • Nomad 1.9.1
  • Vault 1.15

vault secrets list -format=json | jq ‘.[“kv/”].options.version’
“2”

  • ACL
New Style

path “kv/data/docker/nomad/{{identity.entity.aliases.auth_jwt_xxxxxxx.metadata.nomad_namespace}}/{{identity.entity.aliases.auth_jwt_xxxxxxx.metadata.nomad_job_id}}/*” {
capabilities = [“read”]
}

path “kv/data/docker/nomad/{{identity.entity.aliases.auth_jwt_xxxxxxx.metadata.nomad_namespace}}/{{identity.entity.aliases.auth_jwt_xxxxxxx.metadata.nomad_job_id}}” {
capabilities = [“read”]
}

path “kv/metadata/data/docker/nomad/{{identity.entity.aliases.auth_jwt_xxxxxxx.metadata.nomad_namespace}}/*” {
capabilities = [“list”]
}

path “kv/metadata/*” {
capabilities = [“list”]
}
  • Nomad Server
...
  "vault": {
    "enabled": true,
    "create_from_role": "nomad-cluster",
    "ca_file": "/etc/ssl/private/example.crt",
    "cert_file": "/etc/ssl/private/example.crt",
    "key_file": "/etc/ssl/private/example.com.key",
    "address": "https://fra-corp-vault.example.com",
    "tls_skip_verify": false,
    "default_identiy": {"aud": ["production"],
    "ttl": "30m"
     }
  },
  • Nomad Agent
  "vault": {
    "enabled": true,
    "address": "https://fra-corp-vault.example.com",
    "ca_file": "/etc/ssl/private/example.crt",
    "cert_file": "/etc/ssl/private/example.crt",
    "key_file": "/etc/ssl/private/example.com.key",
    "jwt_auth_backend_path": "jwt-nomad/"
},
  • Example hcl Job:
job "mongo" {
  namespace = "corp-services"

  group "db" {
    network {
      port "db" {
        static = 27017
      }
    }

    service {
      provider = "nomad"
      name     = "mongo"
      port     = "db"
    }

    task "mongo" {
      driver = "docker"

      config {
        image = "mongo:7"
        ports = ["db"]
      }
       vault {}

      template {
        data        = <<EOF
MONGO_INITDB_ROOT_USERNAME=root
MONGO_INITDB_ROOT_PASSWORD={{with secret "kv/data/docker/nomad/corp-services/mongo/config"}}{{.Data.data.root_password}}{{end}}
EOF
        destination = "secrets/env"
        env         = true
      }
    }
  }
}

when I try this .. I get:

nomad job run test.hcl
Error submitting job: Unexpected response code: 500 (1 error occurred:
	* Task mongo has a Vault block with an empty list of policies)

If I drop vault {} from the HCL, then jobs gets submitted, but permission denied while access to kv/data/docker/nomad/corp-services/mongo/config

No idea, what is wrong.

Vault logs, looks like this:

{"time":"2025-12-02T19:17:37.830747871Z","type":"response","auth":{"token_type":"default"},"request":{"id":"f330b7d1-4c1d-532f-535a-d8fc153aed03","operation":"read","mount_point":"kv/","mount_type":"kv","mount_running_version":"v0.16.1+builtin","mount_class":"secret","namespace":{"id":"root"},"path":"kv/data/docker/nomad/corp-services/mongo/config","remote_address":"192.168.43.70","remote_port":45586},"response":{"mount_point":"kv/","mount_type":"kv","mount_running_plugin_version":"v0.16.1+builtin","mount_class":"secret","data":{"error":"hmac-sha256:f80c61f345595d14f51d0907ce15c9517e53771c65d69094e8a1706f7fe3be60"}},"error":"1 error occurred:\n\t* permission denied\n\n"}

pretty strange I find this ID with root and namespace .

What do I miss?

cu denny

Update

I guess, I’ve found it: default_identiy vs. default_identity

Yes, it works now. Was really a typo.