Register Nomad namespace service with Consul namespace

Nomad
1

Hey folks, we’re well into our Enterprise adoption, and this is a relatively minor thing, but I’m having trouble figuring out if it’s possible to have Nomad register server namespaces with Consul namespace to provide auto-discovery for self-hosted Nomad Clients.

That’s probably not too clear? Some background:

Previously we had Nomad clients join servers via

  "client": {
    "enabled": true,
    "meta": {
      "service_host": true,
      "jenkins_master": true
    },
    "network_interface": "ens3",
    "options": {
      "docker.auth.config": "/etc/docker/dockercfg-tucows.json",
      "docker.cleanup.image.delay": "3h",
      "driver.raw_exec.enable": "1",
      "driver.whitelist": "docker,raw_exec"
    }
  },
  "consul": {
    "address": "REDACTED",
    "auto_advertise": true,
    "ca_file": "/etc/nomad.d/certs/ca.crt",
    "cert_file": "/etc/nomad.d/certs/server.crt",
    "client_auto_join": true,
    "client_service_name": "nomad-client",
    "key_file": "/etc/nomad.d/certs/server.key",
    "server_auto_join": true,
    "server_service_name": "nomad",
    "ssl": true,
    "token": "REDACTED"
  },

on the client and

  "consul": {
    "address": "REDACTED",
    "auto_advertise": true,
    "ca_file": "/etc/nomad.d/certs/ca.crt",
    "cert_file": "/etc/nomad.d/certs/server.crt",
    "client_auto_join": true,
    "client_service_name": "nomad-client",
    "key_file": "/etc/nomad.d/certs/server.key",
    "server_auto_join": true,
    "server_service_name": "nomad",
    "ssl": true,
    "token": "REDACTED"
  },
  "data_dir": "/var/lib/nomad",
  "datacenter": "bra2",
  "enable_syslog": true,
  "log_level": "DEBUG",
  "server": {
    "bootstrap_expect": 3,
    "enabled": true,
    "encrypt": "REDACTED",
    "heartbeat_grace": "1m"
  }

on the server. This registered the server as a service with Consul and through some black magic, the clients would automatically connect.

Currently, in order to connect self-hosted clients to Nomad Enterprise servers, we have to

  "enabled" = true

  "node_class" = "${hce_nomad_node_class}"

  "options" = {
    "docker.auth.config" = "/etc/docker/tucows.json"

    "docker.cleanup.image.delay" = "3h"

    "docker.privileged.enabled" = "true"

    "docker.volumes.enabled" = "1"

    "driver.raw_exec.enable" = "1"

    "driver.whitelist" = "docker,raw_exec"
  }

  "servers" = ["nomadsrv01.prod-hashicorp-ent.bra2.tucows.systems", "nomadsrv02.prod-hashicorp-ent.bra2.tucows.systems", "nomadsrv03.prod-hashicorp-ent.bra2.tucows.systems", "nomadsrv04.prod-hashicorp-ent.bra2.tucows.systems", "nomadsrv05.prod-hashicorp-ent.bra2.tucows.systems"]
}

"consul" = {
  "client_auto_join" = false

  "namespace" = "${hce_nomad_namespace}"
}```

I can't find anything on automatically registering the servers so we can avoid that clunky "servers" line.

Any ideas or am I completely loony?
2

Right now you can set the Consul namespace on a cluster-wide basis: consul Stanza - Agent Configuration | Nomad by HashiCorp. We recently shipped support for multiple Vault namespaces in Nomad Enterprise and IIRC we intend to do the same for Consul.

3

Ok cool, so if I understand you correctly currently I cannot associate a Nomad namespace with a Consul namespace, but that feature is coming?

Where might I find this info on support for multiple Vault namespaces in Nomad Enterprise?

Thanks!

4

Ok cool, so if I understand you correctly currently I cannot associate a Nomad namespace with a Consul namespace, but that feature is coming?

Sort of: support for multiple Vault namespaces is at the job level. See: vault Block - Job Specification | Nomad | HashiCorp Developer It’ll most likely be done the same way for Consul, rather than mapping namespaces 1:1 to each other. So a given job’s service will be registered to a specific Consul namespace, rather than the whole Nomad namespace.

Once that feature’s available, if you really wanted to always map a Nomad namespace to a Consul namespace, you could probably enforce it via a Sentinel policy.