Using nomad how can I test local generation of templates before sending a job to server

Nomad
1

hi there.

I don’t code with go regularly, so please forgive the silly question - if I have a job file that uses the template stanza, how can I check the output is what i expect before sending a job to a nomad server?

Here’s what I mean.

I have this job file here for caddy, our reverse proxy server that nomad and a bunch of other services are behind:

job "caddy" {
  /*
  A caddy job file for our observability stack. Should be kept separate from our
  app caddy job, to limit changes bringing down both our apps and our monitoring
  */
  datacenters = ["dc1"]
  type        = "service"

  group "caddy" {
    count = 1

    network {
      mode = "host"
      port "https" {
        static = 443
      }
      // you need to have both ports accessible when going through the
      // handshake process to set up SSL with Caddy. If you do not, then
      // you can not complete the handshake when you liston on the "regular"
      // HTTP port of port 80.
      // More here: https://caddyserver.com/docs/automatic-https
      port "http" {
        static = 80
      }
    }

    restart {
      attempts = 3
      delay    = "20s"
      mode     = "delay"
    }

    task "caddy" {
      driver = "docker"

      template {
        data        = file("./nomad/local/Caddyfile.tpl")
        destination = "/local/Caddyfile"
      }

      config {
        args = [
          "caddy",
          "run",
          "--config",
          "/local/Caddyfile",
          "--adapter",
          "caddyfile",
        ]
        image = "caddy:x.x.x"
        ports = ["https", "http"]
      }

      resources {
        cpu    = 200
        memory = 200
      }

      service {
        provider = "nomad"
        name     = "caddy"
        port     = "https"
        tags     = ["monitoring", "caddy"]
      }
    }
  }
}

One of the services we run is rabbitmq, which exposes a dashboard, that I want to make available as a nomad service, so that I can refer to it when generating templates for caddy. Here’s my rabbitmq jobfile, with the services declared:

job "rabbitmq_server" {
  datacenters = ["dc1"]

  type = "service"

  group "rabbit" {

    count = 1

    volume "persistent_data" {
      type      = "host"
      read_only = false
      source    = "persistent_data"
    }

    network {
      mode = "host"

      port "rabbit" {
        static       = 5672
        host_network = "my-internal-network"
      }

      // used by some CLI tools. See below for more:
      // https://rabbitmq.com/networking.html
      port "rabbit-epmd" {
        static       = 4369
        host_network = "my-internal-network"
      }
      
      // used for management 
      // and for prometheus metrics
      // https://rabbitmq.com/management.html
      port "rabbit-dashboard" {
        static        = 15672
        host_network = "my-internal-network"
      }

    }

    task "rabbit" {
      driver = "docker"

      config {
        image = "rabbitmq:3.12-management"
        ports = ["rabbit"]
      }

      // declare our services
      service {
        provider = "nomad"
        name = "rabbitmq-dashboard"
        port = "rabbit-dashboard"
      }

      service {
        provider = "nomad"
        name = "rabbitmq"
        port = "rabbit"

        // check every 10s that the we can connect over tcp
        check {
          type     = "tcp"
          port     = "rabbit"
          interval = "10s"
          timeout  = "2s"
        }
      }
    
      resources {
        cpu    = 1000 # 1000 MHz
        memory = 1028 # 1024 GB
      }

      volume_mount {
        volume      = "persistent_data"
        destination = "/mnt/persistent-vol-1/nomad/rabbitmq"
        read_only   = false
      }

      constraint {
        attribute = "${attr.unique.hostname}"
        value     = "something.my-org.org"
      }
    }
  }
}

Here’s a caddy file I use.

You don’t need to know about caddy really, just that caddy expects a ip address and port combination like the ones below to reverse proxy properly, and do its handy https thing:

{
	cert_issuer acme
	email dev@my-org.org
}

grafana.my-org.org {
	reverse_proxy 1.2.3.4:3000
}

// what hard coded caddy file values normally look  like
nomad.my-org.org {
	reverse_proxy 10.0.0.2:4646
}

// my attempt at using the service syntax instead of 
// relying on hard coded values

rabbit.greenweb.org {
	{{ range nomadService "rabbitmq-dashboard" }}
	reverse_proxy  {{ .Address }}
	{{ end }}
}

If I run this:

nomad job run  ./path/to/caddy-jobfile.nomad.hcl

Then the only way I know if the templated caddyfile contains the correct values is to see if the job fails after it’s been sent to the server.

Is there an easy way to sanity the templating out of the caddyfile without sending a failing job to the server?

If it helps, I want to be able to see the output of this part specifically in a local file if possible, populated with the values from the nomad server.

rabbit.greenweb.org {
	{{ range nomadService "rabbitmq-dashboard" }}
	reverse_proxy  {{ .Address }}
	{{ end }}
}

How do I do this?

2

Oh boy, I’ve seen the typo now.

I refer to rabbitmq-dashboard in the caddy template, but declare the service as rabbit-dashboard in the job file :man_facepalming:.

I have this working now, but I guess the point still stands - is there a way to check this before I send a poorly formatted job to a server?