Vault AppRole for nomad-cluster not found

Vault
1

I’m having an issue when following the guide on setting up mTLS for Nomad with Vault.

I get an error when starting the Vault Agent and it tries to render the cert templates, saying unknown role: nomad-cluster (see full error log below). I’ve terraformed all the resources the guide sets up, and I can verify that the AppRole exists via the terraform console, so I really don’t understand why this error occurs .

I’ve attached the terraform manifests for my configuration below.

2025-01-05T12:25:59.610+0100 [WARN] (view) vault.pki(pki_int/issue/nomad-cluster->/opt/nomad/rendered-cert-files/client.agent.key): Error making API request.

URL: PUT http://vault.service.consul:8200/v1/pki_int/issue/nomad-cluster
Code: 400. Errors:

* unknown role: nomad-cluster (retry attempt 6 after "8s")

Nomad cluster approle verified exists via terraform console:

> vault_approle_auth_backend_role.nomad_cluster
{
  "backend" = "approle"
  "bind_secret_id" = true
  "id" = "auth/approle/role/nomad-cluster"
  "namespace" = tostring(null)
  "role_id" = "80a34ee3-ebc5-eb89-f441-b042998479e6"
  "role_name" = "nomad-cluster"
  "secret_id_bound_cidrs" = toset([])
  "secret_id_num_uses" = 10
  "secret_id_ttl" = 600
  "token_bound_cidrs" = toset([])
  "token_explicit_max_ttl" = 0
  "token_max_ttl" = 900
  "token_no_default_policy" = false
  "token_num_uses" = 0
  "token_period" = 0
  "token_policies" = toset([
    "tls-policy",
  ])
  "token_ttl" = 600
  "token_type" = "batch"
}

Vault PKI

resource "vault_mount" "root" {
  path                      = "pki"
  type                      = "pki"
  description               = "handles root certificates for nomad mtls"
  default_lease_ttl_seconds = 3600
  max_lease_ttl_seconds     = 87600
}

resource "vault_pki_secret_backend_root_cert" "nomad" {
  backend     = vault_mount.root.path
  type        = "internal"
  common_name = "global.nomad"
  ttl         = 87600
}

resource "vault_mount" "pki_int" {
  path                      = "pki_int"
  type                      = vault_mount.root.type
  description               = "handles intermediate certificates for nomad mtls"
  default_lease_ttl_seconds = 3600
  max_lease_ttl_seconds     = 43800
}

resource "vault_pki_secret_backend_intermediate_cert_request" "nomad" {
  backend     = vault_mount.pki_int.path
  type        = vault_pki_secret_backend_root_cert.nomad.type
  common_name = "global.nomad Intermediate Authority"
}

resource "vault_pki_secret_backend_root_sign_intermediate" "nomad" {
  backend              = vault_mount.root.path
  csr                  = vault_pki_secret_backend_intermediate_cert_request.nomad.csr
  common_name          = vault_pki_secret_backend_intermediate_cert_request.nomad.common_name
  exclude_cn_from_sans = true
  format               = "pem_bundle"
  ttl                  = "43800h"
}

resource "vault_pki_secret_backend_intermediate_set_signed" "nomad" {
  backend     = vault_mount.pki_int.path
  certificate = vault_pki_secret_backend_root_sign_intermediate.nomad.certificate
}

AppRole

resource "vault_auth_backend" "approle" {
  type = "approle"
}

resource "vault_approle_auth_backend_role" "nomad_cluster" {
  backend            = vault_auth_backend.approle.path
  role_name          = "nomad-cluster"
  secret_id_num_uses = 10
  secret_id_ttl      = 600
  token_max_ttl      = 900
  token_policies     = [vault_policy.tls_policy.name]
  token_ttl          = 600
  token_type         = "batch"
}

resource "vault_policy" "tls_policy" {
  name   = "tls-policy"
  policy = data.vault_policy_document.nomad_cluster.hcl
}

data "vault_policy_document" "nomad_cluster" {
  rule {
    path         = "pki_int/issue/nomad-cluster"
    capabilities = ["update"]
    description  = "allow nomad cluster access to update certificates"
  }
}

client.agent.key.tpl template

{{ with pkiCert "pki_int/issue/nomad-cluster" "common_name=void.global.nomad" "ttl=24h" "alt_names=localhost" "ip_sans=127.0.0.1"}}
{{ .Data.Key }}
{{ end }}
2

Okay, after comparing the Nomad mTLS guide to the Consul one I found that the Nomad guide completely omits the fact that you have to set up a Vault role in addition to the AppRole authentication.

That is to say, the Consul guide sets this up, while the Nomad guide is completely missing this part:

vault write pki_int/roles/consul-dc1 \
  allowed_domains="dc1.consul" \
  allow_subdomains=true \
  generate_lease=true \
  max_ttl="720h"

If the equivalent role is created for Nomad then it works, so the guide is incomplete.