Render template variables stored in KV

Consul
1

Hello everyone,

Is it possible using consul-template to render a template stored in consul KV?

I have YAML coming from a current workflow stored in KV. I’m able to render a configuration file from that YAML, but I’d like to interpolate the variables (references to services) present in that YAML.

It looks like consul-templaterb is able to do that, using render_from_string (https://github.com/criteo/consul-templaterb/blob/master/TemplateAPI.md#render_from_stringtemplate_to_render-params

Thanks!
Nicolas

2

Currently consul-template doesn’t support templates stored in Consul’s KV store. It works with local (to it) files. There is nothing theoretically preventing some level of support for storing templates in Consul. Please consider filing a feature request ticket or a PR for this functionality.

You might be able to get what you’re after with consul-template by layering it. Have one consul-template grab that value and render it to a file, then have another consul-template monitor that file and render it again to interpolate those variables.

Hope this helps and thanks for your interest!

3

Thanks for your nice solution! I won’t file a feature request, since it’s only a temporary need and the workaround is sufficient.

4

I also arrived at that, but it doesn’t work.

      # Consul HCL Template
      artifact {
        source      = "http://consul.service.consul:8500/v1/kv/config/templates/consul.hcl.tpl?raw"
        destination = "local"
        headers {
          X-Consul-Token = "${CONSUL_HTTP_TOKEN}"
        }
      }
  
      template {
        data = "{{ key \"config/templates/consul.hcl.tpl\" }}\n# Updated: {{ timestamp }}"
        destination = "local/consul.hcl.tpl"
        change_mode = "noop"  # Don't restart, just update file
      }
  
      template {
        source = "local/consul.hcl.tpl"
        destination = "local/consul.hcl"
      }

The artifact is required to fetch the file such that the 2nd template block can pick up the config via source.

The 1st template block is required to trigger an update to the tpl file (using change_mode = "noop" here).

This only works the first time, where the artifact is grabbed and the 2nd template renders it (artifact is only downloaded on task start, but not restart).

However, on subsequent updates to the tpl on Consul KV, only the tpl file is updated, but the 2nd template doesn’t re-render from it for some reason.

You can see here in the screenshot that the tpl file was updated 8min ago (upon me changing the value in Consul KV), but the hcl file did not register the change and therefore did not re-render:

image
image1006×345 10.9 KB

I even tried manually editing the tpl file at /opt/nomad/alloc/969b4514-c61e-1629-d836-bb2d4b58b896/config-reload/local/consul.hcl.tpl, but the 2nd template still didn’t re-render.

I’m not sure if this is a bug or not. I don’t understand why the 2nd template doesn’t register the file change.